tuieval 0.2.0.dev66__tar.gz → 0.2.0.dev70__tar.gz

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## Unreleased
 
+- `tuieval logs` follows the server log of the model running now and moves on with the run; `logs live` shows each answer as it finishes, `--list` every log.
+- Results tables sort by a click on a column name; a second click reverses. Numbers, times and levels sort by value; empty cells stay last.
 - Nvidia and AMD GPUs on Linux: VRAM from `nvidia-smi`, `amd-smi` or `rocm-smi` sizes models (`gpu_memory_gb` overrides it), runs record peak VRAM, and `tuieval doctor` shows the GPU.
 - Results has four tabs: PTA index, Production readiness, Per question and Failures. The PTA table adds tok/s, TTFT and memory; Per question can show only where models disagree; the settings warning is one line. Scorecard, pairwise, history and test quality stay in `tuieval compare`, `history` and `items`.
 - The run's ETA adds up each pack's answers left × that model's time per answer there (its history, then its own times once it has 5 answers), instead of extrapolating from the first few answers.

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tuieval
-Version: 0.2.0.dev66
+Version: 0.2.0.dev70
 Summary: Evaluate local and frontier LLMs on your own questions: accuracy, speed, tokens and PASS/FAIL verdicts, in the terminal.
 Project-URL: Homepage, https://github.com/ashe-wb/tuieval
 Project-URL: Issues, https://github.com/ashe-wb/tuieval/issues
@@ -144,7 +144,7 @@ See **[docs/models.md](docs/models.md)** for servers, OpenRouter endpoint pinnin
 1. **Pick packs and models** (space ticks; type to filter models by label or tag). Each model shows a short code per use case, e.g. `C✓ S?` (✓ pass, ✗ fail, ? inconclusive; grey = from earlier results). To run only some tests of a pack, highlight it and press `e`. Runs of different tests add up, and the pack can PASS once all its tests have run.
 2. **Pick a tier** (Smoke to check setup, Screen, Certify) and **press `s`**. The line above the buttons shows how many answers that is and roughly how long it will take. For each model, tuieval starts its server (or uses a running one), checks the right model is loaded, runs every selected pack, then stops it.
 3. **Watch the run:** progress with ETA, live score, tok/s, TTFT and memory, the current test with reasoning and answer side by side, recent results with the grader's reason. `k` skips a model, `c` cancels (finished work is kept and resumes next time). `n` sets up more evals while it runs: add them to this run (same tier) or queue them as the next run.
-4. **Press `r` for results:** the PTA index (parsimony (tokens), time and accuracy as bars), production readiness, per question, and failures. `tuieval compare`, `tuieval history` and `tuieval items` have the rest: whether a difference is real, verdict history and test quality.
+4. **Press `r` for results:** the PTA index (parsimony (tokens), time and accuracy as bars), production readiness, per question, and failures; click a column name to sort. `tuieval compare`, `tuieval history` and `tuieval items` have the rest: whether a difference is real, verdict history and test quality.
 
 Other keys: `t` tunes the ticked models' speed flags, `a` adds a model, `m` scans for GGUFs, `p` saves or loads a preset, `x` hides a model. `?` explains any screen.
 
@@ -162,6 +162,8 @@ tuieval history                               # every verdict change over time
 tuieval compare --speed --pairwise --failures # scorecards
 tuieval selftest                              # check every test's reference and wrong answers
 tuieval items                                 # tests that don't separate models or look broken
+tuieval logs                                  # follow the server log of the model running now
+tuieval logs live                             # each answer as it finishes (--list: every log)
 tuieval capture logs/live/<file> --pack X     # turn a real failure into a new test
 tuieval regrade                               # re-score stored answers after changing a grader
 tuieval regrade --only <model> --packs X      # just one model's results (and packs)

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/README.md

@@ -118,7 +118,7 @@ See **[docs/models.md](docs/models.md)** for servers, OpenRouter endpoint pinnin
 1. **Pick packs and models** (space ticks; type to filter models by label or tag). Each model shows a short code per use case, e.g. `C✓ S?` (✓ pass, ✗ fail, ? inconclusive; grey = from earlier results). To run only some tests of a pack, highlight it and press `e`. Runs of different tests add up, and the pack can PASS once all its tests have run.
 2. **Pick a tier** (Smoke to check setup, Screen, Certify) and **press `s`**. The line above the buttons shows how many answers that is and roughly how long it will take. For each model, tuieval starts its server (or uses a running one), checks the right model is loaded, runs every selected pack, then stops it.
 3. **Watch the run:** progress with ETA, live score, tok/s, TTFT and memory, the current test with reasoning and answer side by side, recent results with the grader's reason. `k` skips a model, `c` cancels (finished work is kept and resumes next time). `n` sets up more evals while it runs: add them to this run (same tier) or queue them as the next run.
-4. **Press `r` for results:** the PTA index (parsimony (tokens), time and accuracy as bars), production readiness, per question, and failures. `tuieval compare`, `tuieval history` and `tuieval items` have the rest: whether a difference is real, verdict history and test quality.
+4. **Press `r` for results:** the PTA index (parsimony (tokens), time and accuracy as bars), production readiness, per question, and failures; click a column name to sort. `tuieval compare`, `tuieval history` and `tuieval items` have the rest: whether a difference is real, verdict history and test quality.
 
 Other keys: `t` tunes the ticked models' speed flags, `a` adds a model, `m` scans for GGUFs, `p` saves or loads a preset, `x` hides a model. `?` explains any screen.
 
@@ -136,6 +136,8 @@ tuieval history                               # every verdict change over time
 tuieval compare --speed --pairwise --failures # scorecards
 tuieval selftest                              # check every test's reference and wrong answers
 tuieval items                                 # tests that don't separate models or look broken
+tuieval logs                                  # follow the server log of the model running now
+tuieval logs live                             # each answer as it finishes (--list: every log)
 tuieval capture logs/live/<file> --pack X     # turn a real failure into a new test
 tuieval regrade                               # re-score stored answers after changing a grader
 tuieval regrade --only <model> --packs X      # just one model's results (and packs)

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/src/tuieval/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.2.0.dev66'
-__version_tuple__ = version_tuple = (0, 2, 0, 'dev66')
+__version__ = version = '0.2.0.dev70'
+__version_tuple__ = version_tuple = (0, 2, 0, 'dev70')
 
 __commit_id__ = commit_id = None

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/src/tuieval/cli.py

@@ -18,6 +18,7 @@ Day to day
   tuieval report              the verdicts with their evidence, as a markdown file
   tuieval history             every verdict change, newest first
   tuieval items               tests that don't separate models or look broken
+  tuieval logs                follow the server log of the model running now (live: its answers)
   tuieval capture LOG --pack  turn a real failure (logs/live/*.txt) into a new test
   tuieval regrade             re-score stored answers after changing a grader (--only, --packs)
   tuieval list                the models tuieval knows, hidden ones separately (--packs: the packs)
@@ -85,6 +86,9 @@ def main(argv=None):
     if cmd == "watch":
         from . import watch_proxy
         return watch_proxy.main(rest)
+    if cmd == "logs":
+        from . import logs
+        return logs.main(rest)
     from . import run_evals
     if cmd in run_evals.COMMANDS:
         return run_evals.COMMANDS[cmd](rest)

tuieval-0.2.0.dev70/src/tuieval/logs.py

@@ -0,0 +1,220 @@
+"""tuieval logs: follow the server log of the model running now, without naming it.
+
+    tuieval logs              the newest server log, then every log that grows (a header says
+                              which model each part is from), so it moves on with the run
+    tuieval logs live         each answer (reasoning and answer, from logs/live/) as it finishes
+    tuieval logs --list       every model's server log, newest first
+    tuieval logs MODEL        one model's log (any unambiguous part of its label)
+
+Read-only: it never touches servers or results. Only servers tuieval starts write a log here;
+OpenRouter and servers that were already running don't.
+"""
+import argparse
+import datetime
+import os
+import re
+import sys
+import time
+
+from . import workspace
+
+POLL_S = 0.5
+
+
+def server_dir():
+    return workspace.path("logs", "server")
+
+
+def live_dir():
+    return workspace.path("logs", "live")
+
+
+def _files(d, ext):
+    """{path: (size, mtime)} for the files in d with this extension (none if d doesn't exist)."""
+    out = {}
+    try:
+        names = os.listdir(d)
+    except OSError:
+        return out
+    for n in names:
+        if n.endswith(ext):
+            p = os.path.join(d, n)
+            try:
+                st = os.stat(p)
+            except OSError:
+                continue
+            out[p] = (st.st_size, st.st_mtime)
+    return out
+
+
+def _label(path):
+    return os.path.basename(path)[:-len(".log")]
+
+
+def _tail_offset(path, lines):
+    """Where the last `lines` lines of a file start."""
+    with open(path, "rb") as f:
+        f.seek(0, os.SEEK_END)
+        end = pos = f.tell()
+        found, chunk = 0, 8192
+        while pos > 0 and found <= lines:
+            pos = max(0, pos - chunk)
+            f.seek(pos)
+            found = f.read(end - pos).count(b"\n")
+        if pos == 0 and found <= lines:
+            return 0
+        f.seek(pos)
+        data = f.read(end - pos)
+    cut = len(data)
+    for _ in range(lines + 1):
+        cut = data.rfind(b"\n", 0, cut)
+    return pos + cut + 1
+
+
+class Follower:
+    """Follows server logs. `poll()` returns the text written since the last poll, with a
+    `── label ──` header whenever it comes from another log than the text before it.
+    only: follow just this log. Otherwise every log in the folder, including new ones."""
+
+    def __init__(self, folder, lines=40, only=None):
+        self.folder, self.only = folder, only
+        self.offsets = {p: size for p, (size, _) in _files(folder, ".log").items()}
+        self.last = None        # the log the previous text came from
+        self.held = {}          # path -> size at the last poll, for a line still being written
+        start = only or max(self.offsets, key=lambda p: os.path.getmtime(p), default=None)
+        if start and start in self.offsets:
+            self.offsets[start] = _tail_offset(start, lines)
+
+    def poll(self):
+        out = []
+        files = _files(self.folder, ".log")
+        if self.only:
+            files = {p: v for p, v in files.items() if p == self.only}
+        for path, (size, _) in sorted(files.items(), key=lambda kv: kv[1][1]):   # oldest change first
+            off = self.offsets.setdefault(path, 0)
+            if size < off:          # truncated or replaced: read it from the start
+                off = 0
+            if size == off:
+                continue
+            with open(path, "rb") as f:
+                f.seek(off)
+                data = f.read(size - off)
+            cut = data.rfind(b"\n") + 1
+            if not cut:             # no complete line yet: wait one poll for the rest of it
+                if self.held.get(path) != size:
+                    self.held[path] = size
+                    continue
+                cut = len(data)
+            self.held.pop(path, None)
+            self.offsets[path] = off + cut
+            if path != self.last:
+                out.append(f"── {_label(path)} ──\n")
+                self.last = path
+            out.append(data[:cut].decode("utf-8", "replace"))
+        return "".join(out)
+
+
+FOOTER = re.compile(r"^=== .*finish=.* ===$", re.M)
+
+
+class LiveFollower:
+    """New answers in logs/live/ as they're written: each one from its reasoning on (the prompt is
+    in the file), under a header with the file name."""
+
+    def __init__(self, folder, last=1):
+        self.folder = folder
+        files = sorted(_files(folder, ".txt").items(), key=lambda kv: kv[1][1])
+        self.seen = {p for p, _ in files[:max(0, len(files) - last)]}
+
+    def poll(self):
+        out = []
+        for path, (_, mtime) in sorted(_files(self.folder, ".txt").items(), key=lambda kv: kv[1][1]):
+            if path in self.seen:
+                continue
+            try:
+                with open(path, errors="replace") as f:
+                    text = f.read()
+            except OSError:
+                continue
+            if not FOOTER.search(text) and time.time() - mtime < 2:   # still being written
+                continue
+            self.seen.add(path)
+            i = text.find("=== REASONING ===")
+            out.append(f"── {os.path.basename(path)[:-4]} ──\n{text[i:] if i >= 0 else text}\n")
+        return "".join(out)
+
+
+def _ago(mtime):
+    s = max(0, int(time.time() - mtime))
+    if s < 60:
+        return f"{s}s ago"
+    if s < 3600:
+        return f"{s // 60} min ago"
+    if s < 86400:
+        return f"{s // 3600} h ago"
+    return datetime.datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M")
+
+
+def _size(n):
+    for unit in ("B", "KB", "MB", "GB"):
+        if n < 1024 or unit == "GB":
+            return f"{n:.0f} {unit}" if unit == "B" else f"{n:.1f} {unit}"
+        n /= 1024
+
+
+def list_logs(folder):
+    files = sorted(_files(folder, ".log").items(), key=lambda kv: -kv[1][1])
+    if not files:
+        return f"No server logs yet in {os.path.relpath(folder)}/."
+    w = max(len(_label(p)) for p, _ in files)
+    return "\n".join(f"{_label(p):<{w}}  {_size(size):>9}  {_ago(mtime)}" for p, (size, mtime) in files)
+
+
+def find_log(folder, name):
+    """The log whose label is `name`, or the only one containing it. Raises SystemExit otherwise."""
+    labels = {_label(p): p for p in _files(folder, ".log")}
+    if name in labels:
+        return labels[name]
+    hits = sorted(l for l in labels if name.lower() in l.lower())
+    if len(hits) == 1:
+        return labels[hits[0]]
+    if not hits:
+        sys.exit(f"No server log matches {name!r}; tuieval logs --list shows them.")
+    sys.exit(f"{name!r} matches {len(hits)} logs; say which:\n  " + "\n  ".join(hits))
+
+
+def follow(follower, waiting):
+    """Print what the follower finds until ctrl-c."""
+    said = False
+    try:
+        while True:
+            text = follower.poll()
+            if text:
+                sys.stdout.write(text)
+                sys.stdout.flush()
+                said = True
+            elif not said:
+                print(waiting, flush=True)
+                said = True
+            time.sleep(POLL_S)
+    except KeyboardInterrupt:
+        return 0
+
+
+def main(argv):
+    p = argparse.ArgumentParser(prog="tuieval logs", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter,
+                                epilog="ctrl-c stops following (a run going on isn't affected).")
+    p.add_argument("what", nargs="?", help="'live' for answers, or a model (any unambiguous part of its label)")
+    p.add_argument("--list", action="store_true", help="every model's server log, newest first")
+    p.add_argument("-n", "--lines", type=int, default=40, help="lines shown before following (default 40)")
+    a = p.parse_args(argv)
+    if a.list:
+        print(list_logs(server_dir()))
+        return 0
+    if a.what == "live":
+        return follow(LiveFollower(live_dir()),
+                      f"Waiting for answers in {os.path.relpath(live_dir())}/ (ctrl-c stops)")
+    only = find_log(server_dir(), a.what) if a.what else None
+    return follow(Follower(server_dir(), a.lines, only),
+                  f"Waiting for {'that model' if only else 'a server'} to log in "
+                  f"{os.path.relpath(server_dir())}/ (ctrl-c stops)")

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/src/tuieval/tui.py

@@ -39,6 +39,8 @@ from textual.widgets.option_list import Option
 from textual.widgets.selection_list import Selection
 
 import yaml
+from rich.cells import cell_len
+from rich.text import Text
 
 from . import compare
 from . import engine
@@ -181,6 +183,79 @@ def fmt_secs(secs):
     return f"{secs // 60}m{secs % 60:02d}s" if secs >= 60 else f"{secs}s"
 
 
+# -- sorting Results tables by a click on a column name
+SORT_ARROWS = (" ▲", " ▼")
+SORT_RANKS = {"EASY": 1, "MEDIUM": 2, "HARD": 3, "FAIL": 1, "INCONCLUSIVE": 2, "PASS": 3,
+              "DOESN'T FIT": float("inf")}
+
+
+class SortCell(Text):
+    """A table cell that sorts by its `sort_value` instead of what it shows."""
+
+
+def sort_cell(markup, value):
+    t = Text.from_markup(markup)
+    cell = SortCell(t.plain, spans=t.spans)
+    cell.sort_value = value
+    return cell
+
+
+def _sort_number(tok):
+    """1,234 · 3.1k · 2.4× · 12/20 · 4m10s · 1h02m · 12.5s as a number, else None."""
+    tok = tok.rstrip("★×%").replace(",", "")
+    if m := re.fullmatch(r"(\d+)/(\d+)", tok):    # answers right: the share, then the count
+        a, b = int(m[1]), int(m[2])
+        return (a / b if b else 0) + a * 1e-9
+    if (m := re.fullmatch(r"(?:(\d+)h)?(?:(\d+)m)?(?:(\d+(?:\.\d+)?)s)?", tok)) and any(m.groups()):
+        h, mi, s = m.groups()
+        return int(h or 0) * 3600 + int(mi or 0) * 60 + float(s or 0)
+    if m := re.fullmatch(r"(-?\d+(?:\.\d+)?)(k?)", tok):
+        return float(m[1]) * (1000 if m[2] else 1)
+    return None
+
+
+def sort_value(cell):
+    """What a Results cell sorts by: the number it shows, a rank for levels and verdicts, else its
+    text; None (always last) for an empty cell."""
+    if isinstance(cell, SortCell):
+        return cell.sort_value
+    plain = (cell.plain if isinstance(cell, Text) else Text.from_markup(str(cell)).plain).strip()
+    if plain.lower() in ("", "-", "no data", "unrated"):
+        return None
+    if plain.upper() in SORT_RANKS:
+        return SORT_RANKS[plain.upper()]
+    first = re.sub(r"^(OK|TOO SLOW) ", "", plain).split()[0]   # latency: by its p90 seconds
+    n = _sort_number(first)
+    return plain.casefold() if n is None else n
+
+
+class _Desc:
+    """Reverses the order of what it wraps (so empty cells stay last when sorting descending)."""
+    __slots__ = ("v",)
+
+    def __init__(self, v):
+        self.v = v
+
+    def __eq__(self, other):
+        return self.v == other.v
+
+    def __lt__(self, other):
+        return other.v < self.v
+
+
+def sort_key(cell, reverse=False):
+    v = sort_value(cell)
+    if v is None:
+        return (1,)
+    return (0, isinstance(v, str), _Desc(v) if reverse else v)
+
+
+def column_name(label):
+    """A column's label without the sort arrow."""
+    text = label.plain if isinstance(label, Text) else str(label)
+    return next((text[:-len(a)] for a in SORT_ARROWS if text.endswith(a)), text)
+
+
 class StreamView(TextArea):
     """Read-only, word-wrapped text that grows as tokens stream in. Scroll it with the mouse; it follows
     new text only while scrolled to the bottom."""
@@ -340,6 +415,8 @@ with the model's reasoning and answer as they stream, then recent results with t
   Per question           every question, model by model (filters: pack, level, Models disagree)
   Failures               every wrong answer with the grader's reason; enter opens it in full
 
+Click a column name to sort a table by it; click it again to reverse. Empty cells stay last.
+
 More in the terminal: tuieval compare (--pairwise: is a difference real), tuieval history,
 tuieval items (test quality).
 
@@ -2408,7 +2485,7 @@ class ResultsScreen(Screen):
                 yield DataTable(id="pq-summary", zebra_stripes=True, cursor_type="none")
                 yield Static(id="pq-legend")
                 yield DataTable(id="per-question", zebra_stripes=True, cursor_type="cell")
-            with TabPane("Failures"):
+            with TabPane("Failures", id="tab-failures"):
                 yield Static("[dim]Enter on a row shows the whole answer, its reasoning and what was checked.[/dim]")
                 yield DataTable(id="failures", zebra_stripes=True, cursor_type="row")
         yield Footer()
@@ -2417,6 +2494,8 @@ class ResultsScreen(Screen):
         self.rows, self.infos, self.partial_rows, self.superseded = [], [], [], set()
         if not hasattr(self.app, "pq_models"):
             self.app.pq_models = set()
+        if not hasattr(self.app, "result_sorts"):
+            self.app.result_sorts = {}   # table id -> (column name, descending), kept for this session
         self.fill()
 
     def on_screen_resume(self):
@@ -2488,15 +2567,17 @@ class ResultsScreen(Screen):
         f.add_columns("Model", "Level", "Test", "Reason")
         # compare.failures lists the failed rows in order, so they line up with fail_rows
         self.fail_rows = [r for r in rows if not r["ok"]]
-        for model, test, reason, level in compare.failures(rows):
-            f.add_row(model, diff_badge(level), test, rich_escape(reason[:130]))
+        for i, (model, test, reason, level) in enumerate(compare.failures(rows)):
+            f.add_row(model, diff_badge(level), test, rich_escape(reason[:130]), key=str(i))
         self.fill_per_question()
         self.fill_pta()
         self.fill_readiness(results_dir)
+        self.sort_tables()
 
     @on(DataTable.RowSelected, "#failures")
     def failure_selected(self, event):
-        entries = [answer_entry(r) for r in self.fail_rows]
+        # in the order shown, so [ and ] step through them as sorted
+        entries = [answer_entry(self.fail_rows[int(r.key.value)]) for r in event.data_table.ordered_rows]
         i = event.cursor_row
         if not entries[i]:
             self.notify("This result is in the old format and has no stored answer.", severity="warning")
@@ -2507,7 +2588,6 @@ class ResultsScreen(Screen):
     # -- per question: tokens and time for every model on the same question
     @on(Select.Changed, "#pq-pack")
     @on(Select.Changed, "#pq-level")
-    @on(Select.Changed, "#pq-sort")
     @on(Checkbox.Changed, "#pq-partial")
     @on(Checkbox.Changed, "#pq-shared")
     @on(Checkbox.Changed, "#pq-disagree")
@@ -2515,6 +2595,45 @@ class ResultsScreen(Screen):
         if getattr(self, "rows", None) is not None:
             self.fill_per_question()
 
+    @on(Select.Changed, "#pq-sort")
+    def pq_sort_changed(self):
+        """The Sort dropdown takes over from a column sorted by a click."""
+        self.app.result_sorts.pop("per-question", None)
+        self.per_question_changed()
+
+    @on(DataTable.HeaderSelected)
+    def header_selected(self, event):
+        """A click on a column name sorts by it; another click on it reverses the order."""
+        t, name = event.data_table, column_name(event.label)
+        prev = self.app.result_sorts.get(t.id)
+        self.app.result_sorts[t.id] = (name, not prev[1] if prev and prev[0] == name else False)
+        self.sort_table(t)
+
+    def sort_tables(self):
+        """After a table is rebuilt: sort it again by the column picked (if that column is still there)."""
+        for t in self.query(DataTable):
+            self.sort_table(t)
+
+    def sort_table(self, t):
+        name, desc = self.app.result_sorts.get(t.id, (None, False))
+        found = None
+        for col in t.ordered_columns:
+            base = column_name(col.label)
+            label = base + SORT_ARROWS[desc] if base == name else base
+            if col.label.plain != label:
+                col.label = Text(label)
+                w = cell_len(label)   # room for the arrow (a column keeps the width it was added with)
+                col.content_width = max(col.content_width, w)
+                if not col.auto_width:
+                    col.width = max(col.width, w)
+                t._require_update_dimensions = True
+            if base == name:
+                found = col
+        if found is not None:
+            t.sort(found.key, key=lambda cell: sort_key(cell, desc))
+        else:
+            t.refresh()
+
     @on(Button.Pressed, "#pq-models")
     @on(Button.Pressed, "#pta-models")
     def pick_models(self):
@@ -2579,6 +2698,7 @@ class ResultsScreen(Screen):
         for i, h in enumerate(header):   # sized to the contents: added before the rows, a column keeps its header's width
             t.add_column(h, width=max(len(h), *(len(r[i]) for r in table)) if table else None)
         t.add_rows(table)
+        self.sort_table(t)
 
     def fill_per_question(self):
         # a pack being rerun under new questions or settings: its old result file is on its way out,
@@ -2641,18 +2761,21 @@ class ResultsScreen(Screen):
             best = min((c["tokens"] for c in t["cells"].values() if c["tokens"] and c["passed"]), default=None)
             for m in models:
                 c = t["cells"].get(m)
-                cells.append(self.pq_cell(c, best, here) if c else "[dim]-[/dim]")
+                # a model's column sorts by its tokens
+                cells.append(sort_cell(self.pq_cell(c, best, here), c["tokens"] or None) if c else "[dim]-[/dim]")
             spread = t["spread"]["tokens"]
             sp = f"[yellow]{spread:.1f}×[/yellow]" if spread >= 3 else (f"{spread:.1f}×" if len(t["cells"]) > 1 else "")
             desc = t["cells"][next(iter(t["cells"]))]["rows"][0].get("raw", {}).get("description") or t["name"]
             table.add_row(rich_escape(desc[:48]), t["suite"], diff_badge(t["difficulty"]), sp, *cells, key=t["test"])
             self.pq_tests[t["test"]] = t
+        self.sort_table(table)
+        self.sort_table(self.query_one("#pq-summary", DataTable))
         self.query_one("#pq-legend", Static).update(
             f"[dim]{len(tests)} questions. Each cell: result (passes/answers if repeated), median tokens, median seconds. "
             "[green]bold[/green] = fewest tokens among passing answers · [red]cut[/red] = hit max_tokens · "
             "◐ = pack unfinished (not in scores) · dim seconds = hosted, or measured on another machine; tokens compare "
             "across machines, seconds don't. ×tokens = most / fewest tokens across models. Enter on a cell opens the "
-            "answers.[/dim]" + ("\n[yellow]Left out, being rerun with other questions or settings: " + ", ".join(
+            "answers. Click a column name to sort by it (a model's column: by its tokens).[/dim]" + ("\n[yellow]Left out, being rerun with other questions or settings: " + ", ".join(
                 f"{os.path.basename(os.path.dirname(p))} · {os.path.basename(p)[:-5]}" for p in sorted(self.superseded))
                 + ". Only the new run's answers (◐) are shown for these.[/yellow]" if self.superseded else ""))
 

{tuieval-0.2.0.dev66 → tuieval-0.2.0.dev70}/tests/test_tuieval.py

@@ -803,6 +803,18 @@ class PTAIndex(unittest.TestCase):
         picked = pta.index(rows, ["a", "few"])      # picked by hand: compared anyway
         self.assertEqual((picked["questions"], picked["left_out"]), (1, []))
 
+    def test_results_sort_values(self):
+        from tuieval.tui import diff_badge, sort_cell, sort_key, sort_value
+        got = [sort_value(c) for c in ("1,234 ★", "3.1k", "2.4×", "4m10s", "1h02m", "12.5s", diff_badge("hard"),
+                                        "[bold green]PASS[/]", "[green]OK 12s[/] [dim]measured[/dim]",
+                                        "[bold red]doesn't fit[/]", "Top 10 holdings")]
+        self.assertEqual(got, [1234, 3100, 2.4, 250, 3720, 12.5, 3, 3, 12, float("inf"), "top 10 holdings"])
+        self.assertEqual([sort_value(c) for c in ("-", "[dim]-[/dim]", "[dim]no data[/]", "")], [None] * 4)
+        self.assertEqual(sort_value(sort_cell("[green]✓[/green] 3.1k 12s", 3100)), 3100)
+        cells = ["3/4", "-", "10/20", "4/4", "abc"]       # numbers, then text; empty cells last both ways
+        self.assertEqual(sorted(cells, key=sort_key), ["10/20", "3/4", "4/4", "abc", "-"])
+        self.assertEqual(sorted(cells, key=lambda c: sort_key(c, True)), ["4/4", "3/4", "10/20", "abc", "-"])
+
     def test_cli_report_and_tui(self):
         with tempfile.TemporaryDirectory() as tmp:
             ws = os.path.join(tmp, "ws")
@@ -857,8 +869,37 @@ class PTAIndex(unittest.TestCase):
                         app.screen.fill_per_question()
                         await pilot.pause(0.3)
                         alone = pq.row_count
+                        tab = app.screen.query_one(TabbedContent).active
+
+                        async def click_header(t, name):   # a real click on the column name
+                            i = next(i for i, c in enumerate(t.ordered_columns) if c.label.plain.startswith(name))
+                            x = sum(c.get_render_width(t) for c in t.ordered_columns[:i]) + 1
+                            await pilot.click(t, offset=(x, 0))
+                            await pilot.pause(0.2)
+                            return [str(t.get_row_at(r)[0]) for r in range(t.row_count)], \
+                                   [c.label.plain for c in t.ordered_columns]
+                        sorts = {}
+                        pta_t = app.screen.query_one("#pta", DataTable)
+                        sorts["up"], _ = await click_header(pta_t, "answers right")
+                        sorts["down"], sorts["labels"] = await click_header(pta_t, "answers right")
+                        app.pq_models = set()
+                        app.screen.fill_pta()                                   # rebuilt: still sorted
+                        await pilot.pause(0.2)
+                        sorts["kept"] = [str(pta_t.get_row_at(r)[0]) for r in range(pta_t.row_count)]
+                        app.screen.query_one(TabbedContent).active = "tab-failures"
+                        await pilot.pause(0.3)
+                        f = app.screen.query_one("#failures", DataTable)
+                        await click_header(f, "Test")
+                        sorts["tests"], _ = await click_header(f, "Test")
+                        sorts["tests"] = [str(f.get_row_at(r)[2]) for r in range(f.row_count)]
+                        f.focus()
+                        f.move_cursor(row=0)
+                        await pilot.press("enter")                              # opens the row shown, not the first failure
+                        await pilot.pause(0.3)
+                        s = app.screen
+                        sorts["opened"] = s.entries[s.index]["record"]["test"]
                         print(json.dumps({"rows": rows, "bars": "P parsimony" in tri, "disagree": [both, alone],
-                                          "tab": app.screen.query_one(TabbedContent).active}))
+                                          "tab": tab, "sorts": sorts}))
                 asyncio.run(go())
             """)
             p = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True, timeout=300,
@@ -869,6 +910,10 @@ class PTAIndex(unittest.TestCase):
             self.assertEqual(got["disagree"], [4, 0])
             self.assertEqual(got["tab"], "tab-pta")                  # Results opens on the PTA index
             self.assertEqual([(r[0], r[3]) for r in got["rows"]][0], ("good", "4/4"))
+            s = got["sorts"]   # a click on a column name sorts by it, a second click reverses
+            self.assertEqual((s["up"], s["down"], s["kept"]), (["hosted", "good"], ["good", "hosted"], ["good", "hosted"]))
+            self.assertIn("answers right ▼", s["labels"])
+            self.assertEqual((s["tests"][0], s["opened"]), ("apps: d", "d"))
 
 
 class Unavailable(unittest.TestCase):
@@ -1660,5 +1705,88 @@ class Remove(unittest.TestCase):
                 self.assertTrue(os.path.exists(os.path.join(ws, "removed", kept, rel)), rel)
 
 
+
+class Logs(unittest.TestCase):
+    """tuieval logs: follows whichever model's server log is being written, without naming it."""
+
+    def test_follows_the_log_being_written(self):
+        from tuieval import logs
+        with tempfile.TemporaryDirectory() as d:
+            a, b = os.path.join(d, "model-a.log"), os.path.join(d, "model-b.log")
+            write(a, "a old\n")
+            write(b, "".join(f"b {i}\n" for i in range(50)))
+            os.utime(a, (time.time() - 60,) * 2)
+            f = logs.Follower(d, lines=3)
+            self.assertEqual(f.poll(), "── model-b ──\nb 47\nb 48\nb 49\n")   # the newest, its last lines
+            self.assertEqual(f.poll(), "")
+            with open(a, "a") as fh:
+                fh.write("a new\n")                                          # the run moved on to model a
+            self.assertEqual(f.poll(), "── model-a ──\na new\n")
+            with open(b, "a") as fh:
+                fh.write("half a li")                                          # a line still being written
+            self.assertEqual(f.poll(), "")
+            self.assertEqual(f.poll(), "── model-b ──\nhalf a li")             # unchanged since: shown anyway
+            write(os.path.join(d, "model-c.log"), "c starts\n")              # a model that never ran before
+            self.assertEqual(f.poll(), "── model-c ──\nc starts\n")
+            write(a, "fresh\n")                                               # replaced: read from the start
+            self.assertEqual(f.poll(), "── model-a ──\nfresh\n")
+            only = logs.Follower(d, lines=1, only=b)
+            self.assertEqual(only.poll(), "── model-b ──\nb 49\n")             # complete lines first
+            self.assertEqual((only.poll(), only.poll()), ("", "half a li"))     # held one poll, as above
+            with open(a, "a") as fh:
+                fh.write("ignored\n")
+            self.assertEqual(only.poll(), "")
+
+    def test_live_answers(self):
+        from tuieval import logs
+        with tempfile.TemporaryDirectory() as d:
+            def answer(name, done=True, age=0):
+                path = os.path.join(d, name + ".txt")
+                write(path, "MODEL: m\nPROMPT: the question\n\n=== REASONING ===\nthink\n\n=== ANSWER ===\n42\n"
+                      + ("\n=== 10 tokens, 1.0s, finish=stop ===\n" if done else ""))
+                os.utime(path, (time.time() - age,) * 2)
+            answer("20260101_100000_m_pack_old", age=20)
+            answer("20260101_100100_m_pack_last", age=10)
+            f = logs.LiveFollower(d)
+            first = f.poll()
+            self.assertIn("── 20260101_100100_m_pack_last ──\n=== REASONING ===", first)   # the latest one only
+            self.assertNotIn("_old", first)
+            self.assertNotIn("PROMPT", first)
+            answer("20260101_100200_m_pack_new", done=False)                  # still being written
+            self.assertEqual(f.poll(), "")
+            answer("20260101_100200_m_pack_new")
+            self.assertIn("=== ANSWER ===\n42", f.poll())
+
+    def test_cli(self):
+        with tempfile.TemporaryDirectory() as tmp:
+            ws = os.path.join(tmp, "ws")
+            tuieval(tmp, "init", ws)
+            self.assertIn("No server logs yet", tuieval(ws, "logs", "--list").stdout)
+            d = os.path.join(ws, "logs", "server")
+            os.makedirs(d, exist_ok=True)
+            for i, name in enumerate(("coder-q4", "coder-q5", "chat")):
+                write(os.path.join(d, name + ".log"), f"{name} line\n")
+                os.utime(os.path.join(d, name + ".log"), (time.time() - 100 + i,) * 2)
+            listed = tuieval(ws, "logs", "--list").stdout.splitlines()
+            self.assertEqual([l.split()[0] for l in listed], ["chat", "coder-q5", "coder-q4"])   # newest first
+            p = tuieval(ws, "logs", "coder", check=False)
+            self.assertNotEqual(p.returncode, 0)
+            self.assertIn("matches 2 logs", p.stderr)
+            self.assertIn("No server log matches", tuieval(ws, "logs", "nothing", check=False).stderr)
+            # following: the newest log, then whichever one grows; ctrl-c stops it cleanly
+            proc = subprocess.Popen([sys.executable, "-m", "tuieval", "logs", "-n", "1"], stdout=subprocess.PIPE,
+                                    text=True, env=dict(os.environ, TUIEVAL_HOME=ws))
+            try:
+                time.sleep(1.5)
+                with open(os.path.join(d, "coder-q4.log"), "a") as fh:
+                    fh.write("q4 server started\n")
+                time.sleep(1.5)
+            finally:
+                proc.send_signal(__import__("signal").SIGINT)
+                out, _ = proc.communicate(timeout=20)
+            self.assertEqual(proc.returncode, 0)
+            self.assertEqual(out, "── chat ──\nchat line\n── coder-q4 ──\nq4 server started\n")
+
+
 if __name__ == "__main__":
     unittest.main()