tuieval 0.2.0.dev108__tar.gz → 0.2.0.dev116__tar.gz

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/CHANGELOG.md

@@ -2,6 +2,10 @@
 
 ## Unreleased
 
+- Fix: a server that crashes mid-request (the stream ends with no finish reason) was graded as an empty or cut-short answer; it's a server error now, never counted against the model.
+- Dates: answers record `answered_at` (older ones read it from their log name); tests get an added date (an `added` field, or the day tuieval first saw them, in `test-dates.json`). Shown in the answer view, Pick tests, Per question, Failures and Production readiness.
+- Copy reasoning: `y` in the answer view copies its full reasoning, `a` the answer; `y` on the run screen copies the reasoning streamed so far. Uses pbcopy, wl-copy, xclip, xsel or clip, else the terminal (OSC 52).
+- PTA index: click a model's name to remove it from view (every Results tab follows); click it under the bars to bring it back, or show all.
 - The Models heading shows how many are selected, naming any the filter hides; Start asks first when some are hidden.
 - Results → Failures: a Models button filters the list (the same pick as the other tabs), with the number of failures shown.
 - Fix: an answer followed by a stray `</think>` (or a stray code fence after it) was graded as empty; the answer before the tag is graded now. The code grader also reads a doubled ```` ```python ```` fence as one.

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tuieval
-Version: 0.2.0.dev108
+Version: 0.2.0.dev116
 Summary: Evaluate local and frontier LLMs on your own questions: accuracy, speed, tokens and PASS/FAIL verdicts, in the terminal.
 Project-URL: Homepage, https://github.com/ashe-wb/tuieval
 Project-URL: Issues, https://github.com/ashe-wb/tuieval/issues
@@ -66,7 +66,7 @@ While a run goes, you watch each question's reasoning and answer stream in:
 
 ![tuieval's run screen with a model's reasoning streaming in for a coding question](https://raw.githubusercontent.com/ashe-wb/tuieval/main/docs/images/tui-run-reasoning.png)
 
-Afterwards, every answer keeps its reasoning. Enter on a result opens it with the grade and each check, the full reasoning and the answer; `[` and `]` step through the others:
+Afterwards, every answer keeps its reasoning. Enter on a result opens it with the grade and each check, the full reasoning and the answer; `[` and `]` step through the others, `y` copies the reasoning and `a` the answer (on the run screen, `y` copies the reasoning streamed so far):
 
 ![one answer in full: the grade and its checks, the model's reasoning and its code](https://raw.githubusercontent.com/ashe-wb/tuieval/main/docs/images/tui-answer-reasoning.png)
 

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/README.md

@@ -40,7 +40,7 @@ While a run goes, you watch each question's reasoning and answer stream in:
 
 ![tuieval's run screen with a model's reasoning streaming in for a coding question](https://raw.githubusercontent.com/ashe-wb/tuieval/main/docs/images/tui-run-reasoning.png)
 
-Afterwards, every answer keeps its reasoning. Enter on a result opens it with the grade and each check, the full reasoning and the answer; `[` and `]` step through the others:
+Afterwards, every answer keeps its reasoning. Enter on a result opens it with the grade and each check, the full reasoning and the answer; `[` and `]` step through the others, `y` copies the reasoning and `a` the answer (on the run screen, `y` copies the reasoning streamed so far):
 
 ![one answer in full: the grade and its checks, the model's reasoning and its code](https://raw.githubusercontent.com/ashe-wb/tuieval/main/docs/images/tui-answer-reasoning.png)
 

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/docs/writing-packs.md

@@ -63,6 +63,7 @@ min_consistency = 0.95           # share of variant groups where every variant p
   description: refund window       # shown while running and in results
   category: policy                 # optional grouping (Screen runs spread across categories)
   difficulty: medium               # easy | medium | hard: your label, shown in the TUI, never sent to the model
+  added: 2026-10-14                # optional: when the test was written (see Dates below)
   input: |                         # the user message
     A customer bought shoes 20 days ago …
   image: images/receipt.png        # optional: sent as an image (needs vision)
@@ -185,6 +186,10 @@ Every test has a `difficulty` of easy, medium or hard. It's for you, not the mod
 
 A useful convention: **easy** = one rule or a direct read; **medium** = arithmetic, one trap, or two conditions; **hard** = rules in conflict, multi-step reasoning, or misleading data. Once several models have run, `tuieval items` flags labels the results contradict ("easier than labelled": a hard test every model passes; "harder than labelled": an easy test most models fail). `tuieval selftest` warns about tests without a label.
 
+## Dates
+
+Every answer records when it was given (`answered_at`); answers from before that field existed get the time from their log file's name. Every test has an added date: its own `added` field if it has one, otherwise the day tuieval first saw it, kept in `test-dates.json` at the workspace root. Tests that were already there when that file was started get the day of their first answer, or the day the file was started, shown as `≤ 2026-10-01` (on or before). Neither date changes the pack's fingerprint. The TUI shows them in the answer view, Pick tests, Results → Per question (Added), Failures and Production readiness (Answered).
+
 ## Writing tests that separate production-ready models
 
 - **Spec every behaviour you check.** A correct answer written only from the prompt must pass.

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.2.0.dev108'
-__version_tuple__ = version_tuple = (0, 2, 0, 'dev108')
+__version__ = version = '0.2.0.dev116'
+__version_tuple__ = version_tuple = (0, 2, 0, 'dev116')
 
 __commit_id__ = commit_id = None

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/client.py

@@ -118,6 +118,7 @@ def stream_chat(base_url, body, on_delta=None, timeout=1800, stream=None, header
     out = {"answer": "", "reasoning": "", "tool_calls": [], "finish": None, "usage": {}, "timings": {},
            "ttft_s": None, "total_s": None, "error": None}
     reasoning, answer, tools = [], [], {}
+    done = False                      # [DONE] seen
     start = last = time.time()
     try:
         conn.request("POST", prefix + "/v1/chat/completions", body=json.dumps(body),
@@ -142,6 +143,7 @@ def stream_chat(base_url, body, on_delta=None, timeout=1800, stream=None, header
                 continue
             payload = line[5:].strip()
             if payload == "[DONE]":
+                done = True
                 break
             chunk = json.loads(payload)
             if chunk.get("error"):        # OpenRouter reports mid-stream failures this way
@@ -173,6 +175,9 @@ def stream_chat(base_url, body, on_delta=None, timeout=1800, stream=None, header
                 if text:
                     answer.append(text)
                     on_delta and on_delta("answer", text)
+        if not out["error"] and not done and out["finish"] is None:
+            # the stream just ended: a server that crashed or dropped the connection, not the model's answer
+            out["error"] = "connection error: the stream ended before the server finished (no finish reason)"
     except (OSError, http.client.HTTPException, ValueError) as e:
         if stream.cancelled:
             raise Cancelled() from None

tuieval-0.2.0.dev116/src/tuieval/dates.py

@@ -0,0 +1,164 @@
+"""When each answer was given, and when each test was added to its pack.
+
+Answers record `answered_at` (local time with its UTC offset). Older answers have none, but their
+log file name starts with the time the answer finished (logs/live/20261014_143741_…), so they get it
+from that.
+
+A test's date is its `added` field when it has one (tests.yaml: `added: 2026-10-14`). Otherwise
+tuieval notes the day it first sees the test, in test-dates.json at the workspace root. Tests that were
+already there when that file was started get the day of their first answer, or the day the file was
+started, as "on or before" dates: the day they were really written isn't known.
+"""
+import datetime
+import glob
+import json
+import os
+import re
+
+FILE = "test-dates.json"
+_LOG_STAMP = re.compile(r"(\d{8}_\d{6})_")
+_HISTORY = re.compile(r"-\d{8}-\d{6}(\.partial\.jsonl|\.json)$")
+
+
+def now():
+    return datetime.datetime.now().astimezone().isoformat(timespec="seconds")
+
+
+def answered_at(rec):
+    """When an answer finished (an aware datetime), or None if its result doesn't say."""
+    if rec.get("answered_at"):
+        try:
+            return datetime.datetime.fromisoformat(rec["answered_at"])
+        except ValueError:
+            return None
+    m = _LOG_STAMP.match(rec.get("log") or "")
+    if m:
+        try:
+            return datetime.datetime.strptime(m.group(1), "%Y%m%d_%H%M%S").astimezone()
+        except ValueError:
+            return None
+    return None
+
+
+def day(dt):
+    return dt.strftime("%Y-%m-%d") if dt else ""
+
+
+def minute(dt):
+    return dt.strftime("%Y-%m-%d %H:%M") if dt else ""
+
+
+def span(dts):
+    """'2026-10-14', or '2026-10-01 – 10-07' for answers given over several days."""
+    dts = [d for d in dts if d]
+    if not dts:
+        return ""
+    first, last = day(min(dts)), day(max(dts))
+    if first == last:
+        return first
+    return f"{first} – {last[5:] if first[:4] == last[:4] else last}"
+
+
+def _explicit(test):
+    v = test.get("added")
+    if isinstance(v, datetime.datetime):
+        return v.date().isoformat()
+    if isinstance(v, datetime.date):
+        return v.isoformat()
+    if isinstance(v, str):
+        try:
+            return datetime.date.fromisoformat(v.strip()).isoformat()
+        except ValueError:
+            return None
+    return None
+
+
+def first_answers(results_dir):
+    """{(pack, test id): the earliest day it was answered} over every result, history and archive."""
+    first = {}
+
+    def note(pack, rec):
+        dt = answered_at(rec)
+        if pack and dt and rec.get("test"):
+            key, d = (pack, rec["test"]), day(dt)
+            if key not in first or d < first[key]:
+                first[key] = d
+    for path in glob.glob(os.path.join(glob.escape(results_dir), "**", "*.json*"), recursive=True):
+        base = os.path.basename(path)
+        try:
+            if base.endswith(".partial.jsonl"):
+                pack = _HISTORY.sub("", base) if _HISTORY.search(base) else base[:-len(".json.partial.jsonl")]
+                with open(path) as f:
+                    for line in f:
+                        rec = json.loads(line)
+                        if "test" in rec:
+                            note(pack, rec)
+            elif base.endswith(".json"):
+                with open(path) as f:
+                    data = json.load(f)
+                if isinstance(data, dict) and data.get("format") == "evals/1":
+                    for rec in data.get("results", []):
+                        note(data["pack"]["name"], rec)
+        except (OSError, ValueError, KeyError, TypeError):
+            continue
+    return first
+
+
+class AddedDates:
+    """Each test's added date: {(pack, test id): (YYYY-MM-DD, exact)}. Not exact = on or before."""
+
+    def __init__(self, root, packs, results_dir):
+        self.path = os.path.join(root, FILE)
+        try:
+            with open(self.path) as f:
+                saved = json.load(f).get("tests", {})
+            started = True
+        except (OSError, ValueError, AttributeError):
+            saved, started = {}, False
+        self.dates = {}
+        missing = []
+        for name, pack in packs.items():
+            for t in pack.tests:
+                key = (name, t["id"])
+                given = _explicit(t)
+                if given:
+                    self.dates[key] = (given, True)
+                elif f"{name}/{t['id']}" in saved:
+                    s = saved[f"{name}/{t['id']}"]
+                    self.dates[key] = (s["added"], s.get("how") == "seen")
+                else:
+                    missing.append(key)
+        if not missing:
+            return
+        today = datetime.date.today().isoformat()
+        first = first_answers(results_dir)
+        for key in missing:
+            if key in first:           # answered before tuieval noted it: on or before its first answer
+                saved[f"{key[0]}/{key[1]}"] = {"added": first[key], "how": "first answer"}
+            elif started:              # new since the file was started
+                saved[f"{key[0]}/{key[1]}"] = {"added": today, "how": "seen"}
+            else:                      # already there when the file was started
+                saved[f"{key[0]}/{key[1]}"] = {"added": today, "how": "already there"}
+            s = saved[f"{key[0]}/{key[1]}"]
+            self.dates[key] = (s["added"], s["how"] == "seen")
+        try:
+            tmp = self.path + ".tmp"
+            with open(tmp, "w") as f:
+                json.dump({"about": "The day tuieval first saw each test (how: seen), or for tests it found "
+                                    "already there, a day on or before it. A test's own `added` field wins.",
+                           "tests": dict(sorted(saved.items()))}, f, indent=1)
+                f.write("\n")
+            os.replace(tmp, self.path)
+        except OSError:
+            pass                       # a read-only workspace: the dates still show, they just aren't kept
+
+    def get(self, pack, test_id):
+        """(YYYY-MM-DD, exact) or None."""
+        return self.dates.get((pack, test_id))
+
+    def text(self, pack, test_id):
+        """'2026-10-14', '≤ 2026-10-01' (on or before), or ''."""
+        got = self.get(pack, test_id)
+        if not got:
+            return ""
+        return got[0] if got[1] else f"≤ {got[0]}"

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/engine.py

@@ -51,6 +51,7 @@ import urllib.parse
 import urllib.request
 
 from . import client
+from . import dates
 from . import graders
 from . import machines
 from . import packs as packs_mod
@@ -714,6 +715,7 @@ class Engine:
         self.cfg = load_config(self.models_path)
         self.pack_errors = []
         self.packs = packs_mod.load_packs(self.packs_dir, self.pack_errors)
+        self.test_dates = dates.AddedDates(root, self.packs, self.results_dir)
         self._cancel = threading.Event()
         self._main_lane = Lane()
         self._here = threading.local()   # .lane: the lane this thread runs
@@ -766,6 +768,7 @@ class Engine:
         self._add_remote_models()
         self.pack_errors = []
         self.packs = packs_mod.load_packs(self.packs_dir, self.pack_errors)
+        self.test_dates = dates.AddedDates(self.root, self.packs, self.results_dir)
 
     def emit(self, kind, **data):
         try:
@@ -2348,6 +2351,7 @@ class Engine:
                        "severity": g.get("severity"), "answer": res["answer"], "tool_calls": res["tool_calls"],
                        "finish": res["finish"], "reasoning_chars": len(res["reasoning"]),
                        "log": log_name,   # logs/live/<log>: the reasoning, which the result doesn't keep
+                       "answered_at": dates.now(),
                        **({"checks": g["checks"]} if "checks" in g else {}),
                        "machine": self.machine().id,
                        **({"quantization": ep.get("quantization")} if ep else {}),

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/packs.py

@@ -132,7 +132,7 @@ def _read_tests(path):
 # Fields that never reach the model or the grader's decision: editing them doesn't make
 # results outdated.
 NOT_FINGERPRINTED = {"reference", "wrong", "description", "category", "critical", "critical_trial", "group",
-                     "captured_from", "captured_answer", "difficulty"}
+                     "captured_from", "captured_answer", "difficulty", "added"}
 DIFFICULTIES = ("easy", "medium", "hard")
 
 

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/pta.py

@@ -117,12 +117,13 @@ def _blocks(width, v, most):
     return max(1, min(width, round(width * v / most)))
 
 
-def bars(result, width=20, markup=True):
+def bars(result, width=20, markup=True, click=None):
     """Lines with a row per model and a bar each for P, T and A: the at-a-glance view. The P and T bars
     are proportional to tokens and time (the longest is the model that used most; shorter is better),
     with how many times the leanest / fastest model's beside them; the A bar is the share of answers
     right (longer is better, full only when every answer passed), then the share cut off at max_tokens in
-    orange (▓ in plain text) and "(n% truncated)" beside it. markup=False gives plain text."""
+    orange (▓ in plain text) and "(n% truncated)" beside it. markup=False gives plain text; click (an
+    action, e.g. "screen.remove") makes each model's name a link calling it with the row's index."""
     if not result["models"]:
         return []
     name_w = max(len(x["model"]) for x in result["models"])
@@ -133,7 +134,7 @@ def bars(result, width=20, markup=True):
     sub = pad + "".join(f"{better:<{cell_w}}" for _, _, better, *_ in LETTERS)
     lines = [f"[b]{head.rstrip()}[/b]" if markup else head.rstrip(), f"[dim]{sub.rstrip()}[/dim]" if markup else sub.rstrip()]
     most = {m: max((x.get(m) or 0 for x in result["models"]), default=0) for _, _, _, m, _, _ in LETTERS}
-    for x in result["models"]:
+    for row, x in enumerate(result["models"]):
         cells = []
         for k, _, _, measure, shown, color in LETTERS:
             if k == "A":
@@ -155,7 +156,11 @@ def bars(result, width=20, markup=True):
                    if markup else bar + cut_bar + rest)
             cells.append(f"{bar} {num:>6}" + " " * max(0, cell_w - width - 1 - max(6, len(num))))
         name = f"{x['model']:<{name_w}}"
-        lines.append((f"{rich_escape(name) if markup else name}  " + "".join(cells)).rstrip())
+        if markup:
+            pad_w = name_w - len(x["model"])
+            name = (f"[@click={click}({row})]{rich_escape(x['model'])}[/]" + " " * pad_w if click
+                    else rich_escape(name))
+        lines.append((f"{name}  " + "".join(cells)).rstrip())
     return lines
 
 

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/src/tuieval/tui.py

@@ -24,6 +24,8 @@ import glob
 import json
 import os
 import re
+import shutil
+import subprocess
 import threading
 import time
 
@@ -43,6 +45,7 @@ from rich.cells import cell_len
 from rich.text import Text
 
 from . import compare
+from . import dates
 from . import engine
 from . import verdict
 
@@ -289,6 +292,37 @@ class StreamView(TextArea):
         self.call_after_refresh(lambda: self.scroll_to(y=max(0, self.max_scroll_y - from_end), animate=False))
 
 
+CLIPBOARD_TOOLS = (["pbcopy"], ["wl-copy"], ["xclip", "-selection", "clipboard"], ["xsel", "--clipboard", "--input"],
+                   ["clip"])
+
+
+def copy_text(app, text):
+    """Put text on the system clipboard with the first tool here that takes it (pbcopy, wl-copy, xclip, xsel,
+    clip). Without one, the terminal is asked to (OSC 52), which not every terminal allows. True if a tool did."""
+    for cmd in CLIPBOARD_TOOLS:
+        if not shutil.which(cmd[0]):
+            continue
+        try:
+            subprocess.run(cmd, input=text.encode("utf-16" if cmd[0] == "clip" else "utf-8"),
+                           capture_output=True, check=True, timeout=10)
+            return True
+        except (OSError, subprocess.SubprocessError):
+            continue      # e.g. xclip without a display: try the next
+    app.copy_to_clipboard(text)
+    return False
+
+
+def notify_copied(app, what, text, note=""):
+    if not text:
+        app.notify(f"No {what} to copy.", severity="warning")
+    elif copy_text(app, text):
+        app.notify(f"Copied {what}{note}: {len(text):,} chars.")
+    else:
+        app.notify(f"Sent {what}{note} ({len(text):,} chars) to the terminal's clipboard. If nothing pastes, "
+                   "the terminal doesn't allow it: install a clipboard tool (wl-copy, xclip or xsel).",
+                   severity="warning", timeout=10)
+
+
 # ------------------------------------------------------------------ answer detail
 def answer_entry(row):
     """An answer for AnswerScreen from a compare row (None when the row has no stored answer)."""
@@ -451,7 +485,7 @@ with the model's reasoning and answer as they stream, then recent results with t
   PTA index              parsimony (tokens), time and accuracy per model, a bar each: tokens and time
                          as long as they took (shorter is better; 2.4× = 2.4 times the best), accuracy
                          longer is better. Per-answer numbers, tok/s, TTFT and memory in the table.
-                         Results open here.
+                         Results open here. Click a model's name to remove it from view (every tab).
   Production readiness   one verdict per model and use case; the table says why and what to run next
   Per question           every question, model by model (filters: pack, level, Models disagree)
   Failures               every wrong answer with the grader's reason (filter: Models); enter opens it
@@ -511,11 +545,14 @@ class AnswerScreen(ModalScreen):
         Binding("q", "close", "Close", show=False, priority=True),
         Binding("left_square_bracket", "step(-1)", "Previous", priority=True),
         Binding("right_square_bracket", "step(1)", "Next", priority=True),
+        Binding("y", "copy('reasoning')", "Copy reasoning", priority=True),
+        Binding("a", "copy('answer')", "Copy answer", priority=True),
     ]
 
     def __init__(self, entries, index=0, what="answer"):
         super().__init__()
         self.entries, self.index, self.what = entries, index, what
+        self.texts = {}       # what's shown: "reasoning", "answer" -> text, for the copy keys
 
     def check_action(self, action, parameters):
         return len(self.entries) > 1 if action == "step" else True
@@ -550,6 +587,9 @@ class AnswerScreen(ModalScreen):
         self.index = (self.index + delta) % len(self.entries)
         self.show()
 
+    def action_copy(self, what):
+        notify_copied(self.app, what, self.texts.get(what, ""))
+
     def show(self):
         e = self.app.engine
         entry = self.entries[self.index]
@@ -579,11 +619,14 @@ class AnswerScreen(ModalScreen):
                  f"on {rec['machine']}" if rec.get("machine") else "",
                  f"[yellow]ran alongside {', '.join(rec['ran_alongside'])}[/yellow]" if rec.get("ran_alongside") else "",
                  f"seed {rec['seed']}" if "seed" in rec else ""]
+        when = dates.minute(dates.answered_at(rec))
+        added = e.test_dates.text(entry["pack"], rec["test"])
         self.query_one("#answer-head", Static).update(
             f"{mark}  {diff_badge(rec.get('difficulty', 'unrated'))}  [b]{rich_escape(rec.get('description', rec['test']))}[/b]"
             f"{partial}{pos}\n"
             f"[b]{entry['model']}[/b]{quant} · {pack.label if pack else entry['pack']} · {rec['test']} · "
-            f"repeat {rec.get('repeat', 0) + 1}\n"
+            f"repeat {rec.get('repeat', 0) + 1}" + (f" · answered {when}" if when else "")
+            + (f" · test added {added}" if added else "") + "\n"
             + " · ".join(f for f in facts if f))
         grading = [f"[b]Grading:[/b] {rich_escape(rec.get('reason', ''))}"]
         grading += ["  " + line for line in check_lines(rec, test)]
@@ -608,6 +651,7 @@ class AnswerScreen(ModalScreen):
             answer += "\n\n=== TOOL CALLS ===\n" + json.dumps(rec["tool_calls"], indent=2)
         for wid, text in (("#answer-reasoning", reasoning), ("#answer-answer", answer or "(empty)")):
             self.query_one(wid, TextArea).load_text(text)
+        self.texts = {"reasoning": reasoning, "answer": answer}
 
         question, checked = "(test not found)", "(test not found)"
         if test:
@@ -1946,7 +1990,9 @@ class PickTestsScreen(ModalScreen):
         mark = "[b green]\\[✓][/]" if on else "[dim]\\[ ][/dim]"
         tid = f"[b]{rich_escape(t['id'])}[/b]" if on else rich_escape(t["id"])
         note = test_note(t)
-        return f"{mark} {diff_badge(t.get('difficulty', 'unrated'))} {tid}" + (f"  [dim]{rich_escape(note)}[/dim]" if note else "")
+        added = self.app.engine.test_dates.text(self.pack.name, t["id"])
+        return f"{mark} {diff_badge(t.get('difficulty', 'unrated'))} {tid}" + (f"  [dim]{rich_escape(note)}[/dim]" if note else "") \
+            + (f"  [dim]added {added}[/dim]" if added else "")
 
     def fill(self, text):
         ol = self.query_one("#pick-tests", OptionList)
@@ -2163,6 +2209,7 @@ class RunScreen(Screen):
         Binding("c", "cancel_all", "Cancel all", priority=True),
         Binding("f", "toggle_follow", "Pause scroll", priority=True),
         Binding("v", "watch_next", "Next model", priority=True),
+        Binding("y", "copy_reasoning", "Copy reasoning", priority=True),
         Binding("n", "new_run", "More evals", priority=True),
         Binding("r", "results", "Results", priority=True),
         Binding("w", "runs", "Runs", priority=True),
@@ -2182,6 +2229,8 @@ class RunScreen(Screen):
         self.follow = True
         self.buf = {"reasoning": [], "answer": []}
         self.buf_lock = threading.Lock()
+        self.streamed = {"reasoning": [], "answer": []}   # the watched answer so far, untrimmed (for y)
+        self.streamed_whole = False                        # from its start (not joined midway with v)
         self.req_live = {}                    # model label -> its request now: phase, ~tokens, started, last token
         self.started = time.time()
         self.req_started = None
@@ -2344,6 +2393,7 @@ class RunScreen(Screen):
                 v.clear()
         for stream, text in parts.items():
             if text:
+                self.streamed[stream].append(text)
                 self.query_one(f"#{stream}", StreamView).append(text, self.follow)
                 self.req_tokens += max(1, len(text) // 4)
                 if self.req_first is None and self.req_started:
@@ -2364,6 +2414,7 @@ class RunScreen(Screen):
             self.flush_streams()
             self.query_one("#reasoning", StreamView).clear()
             self.query_one("#answer", StreamView).clear()
+            self.streamed, self.streamed_whole = {"reasoning": [], "answer": []}, True
             self.req_started, self.req_tokens, self.req_first = time.time(), 0, None
             job = d["job"]
             rep_note = f" · repeat {d['repeat'] + 1}/{job.repeat}" if job.repeat > 1 else ""
@@ -2731,6 +2782,7 @@ class RunScreen(Screen):
                 v.clear()
         self.query_one("#reasoning", StreamView).clear()
         self.query_one("#answer", StreamView).clear()
+        self.streamed, self.streamed_whole = {"reasoning": [], "answer": []}, False
         self.req_started = None
         self.current = f"Streaming [b]{self.watch}[/b] from its next answer…"
         self.refresh_current()
@@ -2748,6 +2800,12 @@ class RunScreen(Screen):
         self.follow = not self.follow
         self.update_overall()
 
+    def action_copy_reasoning(self):
+        """The streaming answer's reasoning so far, all of it (the pane drops the oldest text when it's long)."""
+        self.flush_streams()
+        notify_copied(self.app, "reasoning", "".join(self.streamed["reasoning"]),
+                      "" if self.streamed_whole else " (from midway: streaming joined after it began)")
+
     @on(DataTable.RowSelected, "#recent")
     def recent_selected(self, event):
         keys = [k for k in event.data_table.rows if k in self.recent_entries]
@@ -2946,14 +3004,15 @@ class ResultsScreen(Screen):
         """Every wrong answer, of the models picked (Models: shared with the other tabs)."""
         f = self.query_one("#failures", DataTable)
         f.clear(columns=True)
-        f.add_columns("Model", "Level", "Test", "Reason")
+        f.add_columns("Model", "Level", "Test", "Answered", "Reason")
         every = {r["model"] for r in self.rows}
         chosen = {m for m in self.app.pq_models if m in every}
         rows = [r for r in self.rows if not chosen or r["model"] in chosen]
         # compare.failures lists the failed rows in order, so they line up with fail_rows
         self.fail_rows = [r for r in rows if not r["ok"]]
         for i, (model, test, reason, level) in enumerate(compare.failures(rows)):
-            f.add_row(model, diff_badge(level), rich_escape(test), rich_escape(reason[:130]), key=str(i))
+            when = dates.day(dates.answered_at(self.fail_rows[i].get("raw") or {}))
+            f.add_row(model, diff_badge(level), rich_escape(test), when, rich_escape(reason[:130]), key=str(i))
         self.query_one("#fail-models", Button).label = f"Models: {len(chosen)} picked" if chosen else "Models: all"
         n = len(self.fail_rows)
         self.query_one("#fail-count", Static).update(
@@ -3032,22 +3091,51 @@ class ResultsScreen(Screen):
         models = sorted(counts, key=lambda m: (-counts[m], m))
 
         def picked(chosen):
-            if chosen is None:
-                return
-            first = chosen and not self.app.pq_models
-            self.app.pq_models = chosen  # kept for this session, so reopening Results keeps it
-            if first:  # a fair comparison needs the same questions: turn it on (you can untick it)
-                with self.query_one("#pq-shared", Checkbox).prevent(Checkbox.Changed):
-                    self.query_one("#pq-shared", Checkbox).value = True
-            # after the picker has closed: a table rebuilt under a modal keeps its header-only widths
-            self.call_after_refresh(self.fill_per_question)
-            self.call_after_refresh(self.fill_pta)
-            self.call_after_refresh(self.fill_failures)
-            self.insights_stale = True
-            if self.query_one(TabbedContent).active == "tab-insights":
-                self.call_after_refresh(self.fill_insights)
+            if chosen is not None:
+                self.set_models(chosen)
         self.app.push_screen(ModelPickScreen(models, counts, self.app.pq_models), picked)
 
+    def set_models(self, chosen):
+        """The models every Results tab compares (none: all of them)."""
+        first = chosen and not self.app.pq_models
+        self.app.pq_models = chosen  # kept for this session, so reopening Results keeps it
+        if first:  # a fair comparison needs the same questions: turn it on (you can untick it)
+            with self.query_one("#pq-shared", Checkbox).prevent(Checkbox.Changed):
+                self.query_one("#pq-shared", Checkbox).value = True
+        # after a picker has closed: a table rebuilt under a modal keeps its header-only widths
+        self.call_after_refresh(self.fill_per_question)
+        self.call_after_refresh(self.fill_pta)
+        self.call_after_refresh(self.fill_failures)
+        self.insights_stale = True
+        if self.query_one(TabbedContent).active == "tab-insights":
+            self.call_after_refresh(self.fill_insights)
+
+    def action_pta_remove(self, row):
+        """A click on a model's name in the PTA bars: leave it out of every tab (click it again below to bring it back)."""
+        shown = [x["model"] for x in getattr(self, "pta_result", {}).get("models", [])]
+        if not 0 <= row < len(shown):
+            return
+        if len(shown) == 1:
+            self.notify("That's the last model shown.", severity="warning")
+            return
+        model = shown[row]
+        self.app.pta_removed = [m for m in getattr(self.app, "pta_removed", []) if m != model] + [model]
+        self.set_models((self.app.pq_models or set(shown)) - {model})
+
+    def action_pta_restore(self, row):
+        removed = self.pta_removed_now()
+        if 0 <= row < len(removed):
+            self.app.pta_removed.remove(removed[row])
+            self.set_models(self.app.pq_models | {removed[row]})
+
+    def action_pta_restore_all(self):
+        self.app.pta_removed = []
+        self.set_models(set())
+
+    def pta_removed_now(self):
+        """Models removed by a click that are still left out (the Models button may have put them back)."""
+        return [m for m in getattr(self.app, "pta_removed", []) if self.app.pq_models and m not in self.app.pq_models]
+
     # -- insights: charts with what to do next (insights.py); built when the tab is first shown
     @on(TabbedContent.TabActivated)
     def tab_activated(self, event):
@@ -3127,7 +3215,16 @@ class ResultsScreen(Screen):
         # none picked: every model with enough answers (pta.index leaves out the rest, and says so)
         result = pta.index(rows, sorted(chosen) or None, pta.memory_gb(self.infos))
         self.query_one("#pta-note", Static).update(pta.scope_note(result, sorted(present)))
-        self.query_one("#pta-bars", Static).update("\n".join(pta.bars(result)) if result["questions"] else "")
+        self.pta_result = result
+        removed = [m for m in self.pta_removed_now() if m in present]
+        self.query_one("#pta-bars", Static).update(
+            "\n".join(pta.bars(result, click="screen.pta_remove")
+                      + ([f"\n[dim]Click a name to remove it. Removed:[/dim] "
+                          + "  ".join(f"[@click=screen.pta_restore({self.pta_removed_now().index(m)})]{rich_escape(m)}[/]"
+                                      for m in removed)
+                          + "  [dim]·[/dim]  [@click=screen.pta_restore_all]show all[/]"] if removed
+                         else ["\n[dim]Click a name to remove it from view.[/dim]"]))
+            if result["questions"] else "")
         self.query_one("#pta-left-out", Static).update(
             "\n".join(f"[dim]{rich_escape(line)}[/dim]" for line in pta.left_out_lines(result)))
         header, table = pta.table(result)
@@ -3193,7 +3290,7 @@ class ResultsScreen(Screen):
         here = self.app.engine.machine().id
         table = self.query_one("#per-question", DataTable)
         table.clear(columns=True)
-        table.add_columns("Test", "Pack", "Level", "×tokens", *models)
+        table.add_columns("Test", "Pack", "Level", "Added", "×tokens", *models)
         self.pq_tests, self.pq_models = {}, models
         for t in tests:
             cells = []
@@ -3205,7 +3302,9 @@ class ResultsScreen(Screen):
             spread = t["spread"]["tokens"]
             sp = f"[yellow]{spread:.1f}×[/yellow]" if spread >= 3 else (f"{spread:.1f}×" if len(t["cells"]) > 1 else "")
             desc = t["cells"][next(iter(t["cells"]))]["rows"][0].get("raw", {}).get("description") or t["name"]
-            table.add_row(rich_escape(desc[:48]), t["suite"], diff_badge(t["difficulty"]), sp, *cells, key=t["test"])
+            added = self.app.engine.test_dates.get(t["suite"], t["test"].split(": ", 1)[-1])
+            added = sort_cell(added[0] if added[1] else f"≤ {added[0]}", added[0]) if added else ""
+            table.add_row(rich_escape(desc[:48]), t["suite"], diff_badge(t["difficulty"]), added, sp, *cells, key=t["test"])
             self.pq_tests[t["test"]] = t
         self.sort_table(table)
         self.sort_table(self.query_one("#pq-summary", DataTable))
@@ -3344,9 +3443,12 @@ class ResultsScreen(Screen):
                                                           "[dim]No verdicts yet: run something from Setup.[/dim]")
         groups = sorted({g for t in table.values() for g in t})
         matrix.add_columns("Model", *groups)
-        detail.add_columns("Model", "Use case", "Pack", "Verdict", "Answers right", "Why")
+        detail.add_columns("Model", "Use case", "Pack", "Verdict", "Answers right", "Answered", "Why")
         right = collections.Counter((r["model"], r["suite"]) for r in self.rows if r["ok"])
         answered = collections.Counter((r["model"], r["suite"]) for r in self.rows)
+        when = collections.defaultdict(list)
+        for r in self.rows:
+            when[(r["model"], r["suite"])].append(dates.answered_at(r.get("raw") or {}))
         lat = verdict.latency_table(self.app.engine, None, table)
         machine_ids = [mc.id for mc in self.app.engine.machines()]
         lt = self.query_one("#latency", DataTable)
@@ -3365,7 +3467,8 @@ class ResultsScreen(Screen):
                     if pv.status != "NO DATA":
                         n = answered[(label, name)]
                         detail.add_row(label, g, name, f"[{style[pv.status]}]{pv.status}[/]",
-                                       f"{right[(label, name)]}/{n}" if n else "-", "; ".join(pv.reasons)[:160])
+                                       f"{right[(label, name)]}/{n}" if n else "-", dates.span(when[(label, name)]),
+                                       "; ".join(pv.reasons)[:160])
 
 
 # ------------------------------------------------------------------ app

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/tests/mock_server.py

@@ -11,6 +11,8 @@ unavailable  HTTP 404 "No endpoints found" for every chat request
 Streams like llama.cpp: reasoning_content then content deltas, usage and timings at the end.
 --stall S: after the first token, S seconds of nothing but keep-alive comments (": OPENROUTER
 PROCESSING"), or of silence with --silent; --stall-before S: the same before the first token.
+--die-after N: the process exits after N stream chunks (0: right after the headers), with no finish
+reason or [DONE], like a server that crashes mid-request.
 An answer longer than the request's max_tokens (one token per word) is cut off with finish
 "length". --max-output N also serves an OpenRouter-style endpoints list with that output limit.
 """
@@ -53,6 +55,7 @@ def reply_for(test, mode):
 class Handler(BaseHTTPRequestHandler):
     answers, mode, model, delay, max_output = {}, "oracle", "mock", 0.0, None
     stall, stall_before, silent = 0.0, 0.0, False
+    die_after = None
 
     def log_message(self, *a):
         pass
@@ -96,10 +99,18 @@ class Handler(BaseHTTPRequestHandler):
         self.send_response(200)
         self.send_header("Content-Type", "text/event-stream")
         self.end_headers()
+        sent = [0]
+
+        def die_if_due():
+            if self.die_after is not None and sent[0] >= self.die_after:
+                self.wfile.flush()
+                os._exit(3)
 
         def send(obj):
+            die_if_due()
             self.wfile.write(f"data: {json.dumps(obj)}\n\n".encode())
             self.wfile.flush()
+            sent[0] += 1
 
         def wait(secs):
             end = time.time() + secs
@@ -123,6 +134,7 @@ class Handler(BaseHTTPRequestHandler):
               "usage": {"prompt_tokens": len(str(content).split()), "completion_tokens": words},
               "timings": {"predicted_n": words, "predicted_per_second": 50.0, "prompt_n": 10,
                           "prompt_per_second": 500.0, "cache_n": 0}})
+        die_if_due()
         self.wfile.write(b"data: [DONE]\n\n")
 
 
@@ -144,10 +156,12 @@ def main(argv=None):
     p.add_argument("--stall", type=float, default=0.0)
     p.add_argument("--stall-before", type=float, default=0.0)
     p.add_argument("--silent", action="store_true")
+    p.add_argument("--die-after", type=int)
     a, _ = p.parse_known_args(argv)   # other flags (e.g. speed knobs under test) are accepted and ignored
     Handler.answers, Handler.mode, Handler.model, Handler.delay = load_answers(a.packs), a.mode, a.model, a.delay
     Handler.max_output = a.max_output
     Handler.stall, Handler.stall_before, Handler.silent = a.stall, a.stall_before, a.silent
+    Handler.die_after = a.die_after
     srv = Server(("127.0.0.1", a.port), Handler)
     print(f"mock server on 127.0.0.1:{a.port} ({a.mode}, {len(Handler.answers)} known questions)", flush=True)
     try:

{tuieval-0.2.0.dev108 → tuieval-0.2.0.dev116}/tests/test_tuieval.py

@@ -2,6 +2,7 @@
 
 Uses a temporary workspace and tests/mock_server.py on a free local port; no model or GPU needed.
 """
+import glob
 import json
 import math
 import os
@@ -452,6 +453,42 @@ class PickTests(unittest.TestCase):
         self.assertIn("'zz'", str(cm.exception))
         self.assertIn("a, b, c, d", str(cm.exception))
 
+    def test_answers_and_tests_are_dated(self):
+        from tuieval import dates, engine, packs
+        before = packs.load_pack(os.path.join(self.ws, "packs", "apps")).fingerprint
+        tests = os.path.join(self.ws, "packs", "apps", "tests.yaml")
+        write(tests, read(tests).replace("{id: a,", "{id: a, added: 2025-03-04,"))
+        self.assertEqual(packs.load_pack(os.path.join(self.ws, "packs", "apps")).fingerprint, before)  # not outdated
+        self.server("oracle")
+        tuieval(self.ws, "run", "--packs", "apps", "--tests", "b")
+        rec = json.loads(read(os.path.join(self.ws, "results", "m", "apps.json")))["results"][0]
+        answered = dates.answered_at(rec)
+        self.assertLess(abs(time.time() - answered.timestamp()), 300)
+        self.assertEqual(rec["answered_at"][:10], time.strftime("%Y-%m-%d"))
+        old = {k: v for k, v in rec.items() if k != "answered_at"}               # older results: from the log name
+        self.assertEqual(dates.answered_at(old).replace(microsecond=0), answered.replace(microsecond=0))
+        self.assertIsNone(dates.answered_at({"log": "unnamed.txt"}))
+
+        def fake_answer(test, day):   # an answer from before the dates file existed
+            write(os.path.join(self.ws, "results", "m", "other.json.partial.jsonl"), json.dumps(
+                {"pack_fingerprint": "x"}) + "\n" + json.dumps({"test": test, "log": f"{day}_101010_m_other_{test}.txt"}) + "\n")
+        fake_answer("c", "20250102")
+        os.remove(os.path.join(self.ws, dates.FILE))
+        e = engine.Engine(root=self.ws)
+        today = time.strftime("%Y-%m-%d")
+        self.assertEqual(e.test_dates.get("apps", "a"), ("2025-03-04", True))     # its own `added` wins
+        self.assertEqual(e.test_dates.get("other", "c"), ("2025-01-02", False))   # on or before its first answer
+        self.assertEqual(e.test_dates.text("apps", "d"), f"≤ {today}")           # already there, never answered
+        write(os.path.join(self.ws, "packs", "apps", "tests.yaml"), read(tests) + "- {id: e, input: apps question e, "
+              "expected: 1, reference: 'ANSWER: 1', difficulty: easy}\n")
+        e.reload()
+        self.assertEqual(e.test_dates.get("apps", "e"), (today, True))            # new since: the day it was seen
+        self.assertEqual(engine.Engine(root=self.ws).test_dates.get("other", "c"), ("2025-01-02", False))  # kept
+        self.assertEqual(dates.span([answered]), answered.strftime("%Y-%m-%d"))
+        self.assertEqual(dates.span([dates.answered_at(old), dates.answered_at({"log": "20250102_101010_x"})]),
+                         f"2025-01-02 – {answered.strftime('%m-%d')}" if answered.year == 2025 else
+                         f"2025-01-02 – {answered.strftime('%Y-%m-%d')}")
+
     def test_partial_runs_add_up_and_force_redoes_only_the_picked(self):
         self.server("oracle")
         tuieval(self.ws, "run", "--packs", "apps", "--tests", "b")
@@ -1130,6 +1167,70 @@ class AddToRun(unittest.TestCase):
         self.assertEqual(fewer, 4)                     # its 4 questions are no longer counted
         self.assertEqual(out["running"], [False, True])
 
+    def test_tui_copies_reasoning_and_answer(self):
+        path = os.path.join(self.ws, "models.toml")
+        write(path, read(path).replace('"--delay", "0.2"', '"--delay", "1.5"'))   # reasoning first, then a wait
+        fake = os.path.join(self.tmp.name, "bin")
+        os.makedirs(fake)
+        write(os.path.join(fake, "pbcopy"), '#!/bin/sh\ncat > "$CLIP_OUT"\n')
+        os.chmod(os.path.join(fake, "pbcopy"), 0o755)
+        clip = os.path.join(self.tmp.name, "clip.txt")
+        code = textwrap.dedent("""
+            import asyncio, json, os, time
+            from tuieval.tui import AnswerScreen, EvalsApp, RunScreen
+            from textual.widgets import SelectionList
+            def pasted():
+                with open(os.environ["CLIP_OUT"]) as f:
+                    text = f.read()
+                os.remove(os.environ["CLIP_OUT"])
+                return text
+            async def go():
+                app = EvalsApp({})
+                async with app.run_test(size=(160, 50)) as pilot:
+                    await pilot.pause()
+                    s = app.screen
+                    s.query_one("#suites", SelectionList).select("apps")
+                    s.selected_models = {"a"}
+                    s.refresh_models()
+                    s.query_one("#tier-certify").value = True
+                    await pilot.pause()
+                    s.action_start()
+                    deadline = time.time() + 180     # CI machines can take long to start servers
+                    while time.time() < deadline:
+                        await pilot.pause(0.1)
+                        if isinstance(app.screen, RunScreen) and app.screen.req_started:
+                            break
+                    run, out = app.screen, {}
+                    await pilot.pause(0.5)            # the reasoning has streamed; the answer waits 1.5 s
+                    await pilot.press("y")
+                    await pilot.pause()
+                    out["live"] = pasted()
+                    deadline = time.time() + 120
+                    while run.running and time.time() < deadline:
+                        await pilot.pause(0.2)
+                    app.push_screen(AnswerScreen(list(run.recent_entries.values())))
+                    await pilot.pause()
+                    for key in "ya":
+                        await pilot.press(key)
+                        await pilot.pause()
+                        out[key] = pasted()
+                    print(json.dumps(out))
+            asyncio.run(go())
+        """)
+        p = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, timeout=600,
+                           env=dict(os.environ, TUIEVAL_HOME=self.ws, TUIEVAL_DETECT_PORTS="", CLIP_OUT=clip,
+                                    PATH=fake + os.pathsep + os.environ["PATH"]))
+        out = json.loads(p.stdout.strip().splitlines()[-1]) if p.stdout.strip() else p.stderr
+        self.assertEqual(out, {"live": "Thinking it over. ", "y": "Thinking it over. ", "a": "ANSWER: 1"}, out)
+
+    def test_copy_falls_back_to_the_terminal(self):
+        from unittest import mock
+        from tuieval.tui import copy_text
+        app = mock.Mock()
+        with mock.patch.dict(os.environ, {"PATH": self.tmp.name}):      # no clipboard tool here
+            self.assertFalse(copy_text(app, "some reasoning"))
+        app.copy_to_clipboard.assert_called_once_with("some reasoning")
+
     def test_tui_counts_ticked_models_and_names_filtered_ones_before_start(self):
         code = textwrap.dedent("""
             import asyncio, json
@@ -1609,6 +1710,23 @@ class PTAIndex(unittest.TestCase):
                         app.screen.fill_pta()                                   # rebuilt: still sorted
                         await pilot.pause(0.2)
                         sorts["kept"] = [str(pta_t.get_row_at(r)[0]) for r in range(pta_t.row_count)]
+                        bars_w = app.screen.query_one("#pta-bars", Static)
+                        shown = [x["model"] for x in app.screen.pta_result["models"]]
+                        await pilot.click(bars_w, offset=(1, 1 + 2 + shown.index("hosted")))   # a click on its name
+                        await pilot.pause(0.3)
+                        clicks = {"removed": [sorted(app.pq_models), [x["model"] for x in app.screen.pta_result["models"]],
+                                              "Removed: hosted" in str(bars_w.render())]}
+                        await pilot.click(bars_w, offset=(1, 1 + 2))                         # the last one stays
+                        await pilot.pause(0.3)
+                        clicks["last"] = sorted(app.pq_models)
+                        app.screen.action_pta_restore(0)
+                        await pilot.pause(0.3)
+                        clicks["restored"] = [sorted(app.pq_models), len(app.screen.pta_result["models"])]
+                        app.screen.action_pta_remove(shown.index("hosted"))
+                        app.screen.action_pta_restore_all()
+                        await pilot.pause(0.3)
+                        clicks["all"] = [sorted(app.pq_models), "Removed" in str(bars_w.render())]
+                        sorts["clicks"] = clicks
                         app.screen.query_one(TabbedContent).active = "tab-failures"
                         await pilot.pause(0.3)
                         f = app.screen.query_one("#failures", DataTable)
@@ -1647,6 +1765,11 @@ class PTAIndex(unittest.TestCase):
             s = got["sorts"]   # a click on a column name sorts by it, a second click reverses
             self.assertEqual((s["up"], s["down"], s["kept"]), (["hosted", "good"], ["good", "hosted"], ["good", "hosted"]))
             self.assertIn("answers right ▼", s["labels"])
+            c = s["clicks"]   # a click on a name in the bars leaves it out of every tab; the last one stays
+            self.assertEqual(c["removed"], [["good"], ["good"], True])
+            self.assertEqual(c["last"], ["good"])
+            self.assertEqual(c["restored"], [["good", "hosted"], 2])
+            self.assertEqual(c["all"], [[], False])
             self.assertEqual((s["tests"][0], s["opened"]), ("apps: d", "d"))
             fails = got["fails"]
             self.assertEqual(fails["all"], [4, "4 failures"])
@@ -2724,6 +2847,25 @@ class StallGuard(unittest.TestCase):
         self.assertFalse(client.is_server_error(res["error"]))   # slow, like any timeout: counts
         self.assertLess(took, 4)
 
+    def test_crash_before_any_token_is_a_server_error(self):
+        from tuieval import client
+        res, _ = self.ask("--die-after", "0", timeout=30)
+        self.assertTrue(client.is_server_error(res["error"]), res["error"])
+        self.assertIn("stream ended", res["error"])
+        self.assertEqual(res["answer"], "")
+
+    def test_crash_mid_answer_is_a_server_error(self):
+        from tuieval import client
+        res, _ = self.ask("--die-after", "2", timeout=30)
+        self.assertTrue(client.is_server_error(res["error"]), res["error"])
+        self.assertIsNone(res["finish"])
+
+    def test_finish_without_done_is_an_answer(self):
+        res, _ = self.ask("--die-after", "99", timeout=30)    # every chunk, then gone before [DONE]
+        self.assertIsNone(res["error"])
+        self.assertEqual(res["finish"], "stop")
+        self.assertIn("42", res["answer"])
+
     def test_run_screen_shows_each_running_models_progress(self):
         from types import SimpleNamespace
         from tuieval import tui
@@ -2868,5 +3010,33 @@ class Insights(unittest.TestCase):
             self.assertEqual(p.stdout.split(), ["True", "True", "True", "True", "bad"], p.stderr[-2000:])
 
 
+class ServerCrash(unittest.TestCase):
+    """A server that crashes mid-request (the stream just ends) never judges the model: the run says the
+    server stopped and records no answer."""
+
+    def test_crash_records_no_answer(self):
+        with tempfile.TemporaryDirectory() as tmp:
+            ws = os.path.join(tmp, "ws")
+            tuieval(tmp, "init", ws)
+            d = os.path.join(ws, "packs", "apps")
+            os.makedirs(d)
+            write(os.path.join(d, "pack.toml"), 'label = "apps"\n[certify]\nrepeat = 1\n[gate]\nmin_accuracy = 0.1\n')
+            write(os.path.join(d, "tests.yaml"), "- {id: a, input: question a, expected: 1, reference: 'ANSWER: 1'}\n")
+            mock = [sys.executable, os.path.join(HERE, "mock_server.py"), "--port", "{port}", "--packs",
+                    os.path.join(ws, "packs"), "--model", "{served_name}", "--die-after", "0"]
+            with open(os.path.join(ws, "models.toml"), "a") as f:
+                f.write(f"\n[servers.mk]\ncmd = {json.dumps(mock)}\nport = {free_port()}\n"
+                        '\n[[models]]\nlabel = "crashy"\nserver = "mk"\nmodel = "crashy"\n')
+            p = tuieval(ws, "run", "--only", "crashy", "--tier", "certify", check=False)
+            self.assertIn("server stopped", p.stdout + p.stderr)
+            self.assertNotIn("Empty answer", p.stdout + p.stderr)
+            rows = []
+            for path in glob.glob(os.path.join(ws, "results", "crashy", "*.json*")):
+                text = read(path)
+                rows += [r for r in (json.loads(text).get("results", []) if path.endswith(".json") else
+                                     [json.loads(l) for l in text.splitlines() if l.strip()]) if "test" in r]
+            self.assertEqual(rows, [])
+
+
 if __name__ == "__main__":
     unittest.main()