mjolnir-cli 0.2.0__tar.gz → 0.4.2__tar.gz

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/.gitignore

@@ -1,8 +1,18 @@
 __pycache__/
 *.py[cod]
 .venv/
+.playwright-mcp/
 .pytest_cache/
 .ruff_cache/
+.env
+# the configs' env sidecars are plain mjolnir knobs (calibration paths, feature
+# flags) and ride with the configs; real secrets stay out
+!configs/**/.env
+# coverage artifacts (the CI test stage writes these; they regenerate)
+.coverage
+.coverage.*
+coverage.xml
+.coverage_html/
 dist/
 build/
 *.egg-info/
@@ -18,6 +28,7 @@ benchmarks/raw/*
 !benchmarks/raw/accuracy/
 benchmarks/raw/accuracy/*/*/
 benchmarks/logs/
+benchmarks/staging/
 task/
 
 # autoresearch: the live ledger is untracked (discards are `git reset --hard`);
@@ -26,3 +37,6 @@ research/ledger.jsonl
 research/ledger.tsv
 # …and the exported runs ARE committed (the generic runs/ rule above would hide them)
 !research/runs/
+/cutlass/
+/fa2/
+/llamacpp/

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/NOTICE

@@ -8,7 +8,7 @@ works listed below; each keeps its own license and copyright notice.
 -------------------------------------------------------------------------------
 1. NVIDIA CUTLASS — CuTe DSL examples                         BSD-3-Clause
 -------------------------------------------------------------------------------
-src/mjolnir/mjolnir/t5_decode.py (the Mjolnir decode-attention kernel) is
+src/mjolnir/mjolnir/arch/thor/decode.py (the Mjolnir decode-attention kernel) is
 derived from
 
   examples/python/CuTeDSL/cute/blackwell/kernel/attention/mixed_input_fmha/

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/PKG-INFO

@@ -1,13 +1,13 @@
 Metadata-Version: 2.5
 Name: mjolnir-cli
-Version: 0.2.0
+Version: 0.4.2
 Summary: Serve, benchmark and document LLMs on NVIDIA Jetson AGX Thor: YAML model configs, the Mjolnir vLLM image, KV-cache calibration, kernel / serving / accuracy benchmarks and auto-rendered reports.
-Project-URL: Homepage, https://github.com/thor-arena/mjolnir
-Project-URL: Documentation, https://github.com/thor-arena/mjolnir/tree/main/docs
-Project-URL: Repository, https://github.com/thor-arena/mjolnir.git
-Project-URL: Issues, https://github.com/thor-arena/mjolnir/issues
-Project-URL: Changelog, https://github.com/thor-arena/mjolnir/releases
-Project-URL: CLI reference, https://github.com/thor-arena/mjolnir/blob/main/docs/cli.md
+Project-URL: Homepage, https://github.com/GoBrrr-ai/mjolnir
+Project-URL: Documentation, https://github.com/GoBrrr-ai/mjolnir/tree/main/docs
+Project-URL: Repository, https://github.com/GoBrrr-ai/mjolnir.git
+Project-URL: Issues, https://github.com/GoBrrr-ai/mjolnir/issues
+Project-URL: Changelog, https://github.com/GoBrrr-ai/mjolnir/releases
+Project-URL: CLI reference, https://github.com/GoBrrr-ai/mjolnir/blob/main/docs/cli.md
 Author-email: Mieszko Syty <mieszko@ms1design.pl>
 License-Expression: Apache-2.0
 License-File: LICENSE
@@ -18,13 +18,11 @@ Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
 Classifier: Operating System :: POSIX :: Linux
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: System :: Benchmark
-Requires-Python: >=3.10
+Requires-Python: >=3.12
 Requires-Dist: jinja2>=3.1.6
 Requires-Dist: pyyaml>=6.0.3
 Requires-Dist: rich>=15.0.0
@@ -36,7 +34,7 @@ Description-Content-Type: text/markdown
 # mjolnir-cli
 
 **Serve, benchmark and document LLMs on the NVIDIA Jetson AGX Thor** with
-[Mjolnir](https://github.com/thor-arena/mjolnir) — a decode-attention kernel
+[Mjolnir](https://github.com/GoBrrr-ai/mjolnir) — a decode-attention kernel
 built for Thor (`sm_110`), a 6-bit KV-cache format for vLLM and a patched vLLM
 image that runs them. This package is the `mjolnir` command that drives all of
 it; the kernel itself ships separately as
@@ -71,17 +69,45 @@ mjolnir serve up                 # serve the default model on the Mjolnir image,
   tests and logs kernel experiments.
 
 Every command, with options and defaults:
-**[the CLI reference](https://github.com/thor-arena/mjolnir/blob/main/docs/cli.md)**
+**[the CLI reference](https://github.com/GoBrrr-ai/mjolnir/blob/main/docs/cli.md)**
 (generated from the commands; `mjolnir <command> --help` prints the same).
 
+## Library use
+
+Two modules are declared a public, stable API — reuse them instead of
+re-implementing the clean-window gate or the bench-history substrate:
+
+- **`mjolnir_cli.gate`** — `wait_for_idle`, `server_load`, `server_counters`,
+  `spec_delta`, the `CleanWindow` context manager, `PreflightError` and the
+  `DEFAULT_*` tuning constants.
+- **`mjolnir_cli.history`** — `append_record`, `load_records` and
+  `SCHEMA_VERSION`; the history-row schema is documented in the module
+  docstring. Rows carry `schema_version` — absence means the legacy (pre-0.3)
+  format — and `load_records` returns rows raw, never normalized.
+  `append_record` files one record per serving key (platform, model, image,
+  config): a re-run merges its cells into the stored record instead of
+  appending a line; records without a serving key are appended verbatim.
+
+Anything else can change without notice; the module docstrings are the API
+reference: [`gate.py`](https://github.com/GoBrrr-ai/mjolnir/blob/main/src/mjolnir-cli/mjolnir_cli/gate.py),
+[`history.py`](https://github.com/GoBrrr-ai/mjolnir/blob/main/src/mjolnir-cli/mjolnir_cli/history.py).
+
+`uv tool install mjolnir-cli` is an isolated CLI install — *not* importable
+from another project. Library use means adding `mjolnir-cli` as a dependency
+of your project (`uv add mjolnir-cli` / `pip install mjolnir-cli`). The
+gate/history import chain is stdlib-only: the CLI dependency stack (typer,
+rich, pyyaml, jinja2) is paid at install time, not import time. Python ≥ 3.12
+is required (pip refuses older interpreters — relevant on JetPack 6-era
+images), and type annotations ship (`py.typed`).
+
 ## Requirements
 
 A Jetson AGX Thor (JetPack 7.x) with Docker and the NVIDIA Container Toolkit
-for anything that touches the GPU; Python ≥ 3.10. Charts need the `plot` extra
+for anything that touches the GPU; Python ≥ 3.12. Charts need the `plot` extra
 (`uv tool install 'mjolnir-cli[plot]'`).
 
 ## Versions
 
 `mjolnir-cli` and `mjolnir-kernel` version independently; the Docker image is
 tagged after the upstream vLLM it is built on (`v0.31.0`). Release notes:
-[GitHub Releases](https://github.com/thor-arena/mjolnir/releases).
+[GitHub Releases](https://github.com/GoBrrr-ai/mjolnir/releases).

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/README.md

@@ -1,7 +1,7 @@
 # mjolnir-cli
 
 **Serve, benchmark and document LLMs on the NVIDIA Jetson AGX Thor** with
-[Mjolnir](https://github.com/thor-arena/mjolnir) — a decode-attention kernel
+[Mjolnir](https://github.com/GoBrrr-ai/mjolnir) — a decode-attention kernel
 built for Thor (`sm_110`), a 6-bit KV-cache format for vLLM and a patched vLLM
 image that runs them. This package is the `mjolnir` command that drives all of
 it; the kernel itself ships separately as
@@ -36,17 +36,45 @@ mjolnir serve up                 # serve the default model on the Mjolnir image,
   tests and logs kernel experiments.
 
 Every command, with options and defaults:
-**[the CLI reference](https://github.com/thor-arena/mjolnir/blob/main/docs/cli.md)**
+**[the CLI reference](https://github.com/GoBrrr-ai/mjolnir/blob/main/docs/cli.md)**
 (generated from the commands; `mjolnir <command> --help` prints the same).
 
+## Library use
+
+Two modules are declared a public, stable API — reuse them instead of
+re-implementing the clean-window gate or the bench-history substrate:
+
+- **`mjolnir_cli.gate`** — `wait_for_idle`, `server_load`, `server_counters`,
+  `spec_delta`, the `CleanWindow` context manager, `PreflightError` and the
+  `DEFAULT_*` tuning constants.
+- **`mjolnir_cli.history`** — `append_record`, `load_records` and
+  `SCHEMA_VERSION`; the history-row schema is documented in the module
+  docstring. Rows carry `schema_version` — absence means the legacy (pre-0.3)
+  format — and `load_records` returns rows raw, never normalized.
+  `append_record` files one record per serving key (platform, model, image,
+  config): a re-run merges its cells into the stored record instead of
+  appending a line; records without a serving key are appended verbatim.
+
+Anything else can change without notice; the module docstrings are the API
+reference: [`gate.py`](https://github.com/GoBrrr-ai/mjolnir/blob/main/src/mjolnir-cli/mjolnir_cli/gate.py),
+[`history.py`](https://github.com/GoBrrr-ai/mjolnir/blob/main/src/mjolnir-cli/mjolnir_cli/history.py).
+
+`uv tool install mjolnir-cli` is an isolated CLI install — *not* importable
+from another project. Library use means adding `mjolnir-cli` as a dependency
+of your project (`uv add mjolnir-cli` / `pip install mjolnir-cli`). The
+gate/history import chain is stdlib-only: the CLI dependency stack (typer,
+rich, pyyaml, jinja2) is paid at install time, not import time. Python ≥ 3.12
+is required (pip refuses older interpreters — relevant on JetPack 6-era
+images), and type annotations ship (`py.typed`).
+
 ## Requirements
 
 A Jetson AGX Thor (JetPack 7.x) with Docker and the NVIDIA Container Toolkit
-for anything that touches the GPU; Python ≥ 3.10. Charts need the `plot` extra
+for anything that touches the GPU; Python ≥ 3.12. Charts need the `plot` extra
 (`uv tool install 'mjolnir-cli[plot]'`).
 
 ## Versions
 
 `mjolnir-cli` and `mjolnir-kernel` version independently; the Docker image is
 tagged after the upstream vLLM it is built on (`v0.31.0`). Release notes:
-[GitHub Releases](https://github.com/thor-arena/mjolnir/releases).
+[GitHub Releases](https://github.com/GoBrrr-ai/mjolnir/releases).

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/mjolnir_cli/__init__.py

@@ -5,4 +5,4 @@ kernel against FlashInfer and FlashAttention 2, and render the
 data-driven README charts — one command surface.
 """
 
-__version__ = "0.2.0"
+__version__ = "0.4.2"

mjolnir_cli-0.4.2/mjolnir_cli/__main__.py

@@ -0,0 +1,10 @@
+"""Run the ``mjolnir`` CLI via ``python -m mjolnir_cli`` — the same entry
+point as the console script (``mjolnir_cli.cli:main``), UnknownPlatform
+handler included."""
+
+import sys
+
+from mjolnir_cli.cli import main
+
+if __name__ == "__main__":
+    sys.exit(main())

{mjolnir_cli-0.2.0 → mjolnir_cli-0.4.2}/mjolnir_cli/accuracy.py

@@ -14,8 +14,9 @@ tool-eval-bench keeps its results in ``./data/benchmarks.sqlite`` and
 under ``benchmarks/raw/accuracy/<run>/<leg>/``, so the database holds exactly
 that leg's runs. :func:`summarize_db` reads it back into the committed
 ``accuracy.json`` (``mjolnir.accuracy/1``): per suite the score, per
-tool-call category the share of points, per scenario the points — what the
-docs table and charts are rendered from.
+tool-call category the share of points, per scenario the points, and the ids
+tool-eval-bench excluded from scoring (``excluded``) — what the docs table
+and charts are rendered from.
 """
 
 from __future__ import annotations
@@ -24,6 +25,7 @@ import json
 import os
 import shlex
 import shutil
+import platform as host_platform  # the record's host field names the machine (platform.node)
 import sqlite3
 import subprocess
 import sys
@@ -31,6 +33,9 @@ import time
 from pathlib import Path
 from statistics import mean
 
+from mjolnir_cli.config import by_model, model_of, model_short
+from mjolnir_cli.platforms import platform_of
+
 FORMAT = "mjolnir.accuracy/1"
 TOOL_REPO = "https://github.com/SeraphimSerapis/tool-eval-bench"
 # pinned: a score is only comparable to one produced by the same scenario set
@@ -94,7 +99,9 @@ def _pct(earned: float, mx: float) -> float | None:
 def summarize_db(db: Path) -> dict:
     """tool-eval-bench's SQLite → ``{"suites": {suite: {...}}, "categories":
     {...}, "scenarios": {...}}``. Several completed runs of a kind (``--trials``)
-    are averaged; each suite keeps its per-run scores."""
+    are averaged; each suite keeps its per-run scores. A scenario the tool
+    excluded from scoring (an infrastructure failure: the probe failed, a
+    timeout) comes back as ``excluded``."""
     if not db.exists():
         return {"suites": {}, "categories": {}, "scenarios": {}, "runs": []}
     con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
@@ -108,13 +115,15 @@ def summarize_db(db: Path) -> dict:
     per_suite: dict[str, list[dict]] = {}
     cats: dict[str, dict] = {}
     scen: dict[str, list[float]] = {}
+    excluded: set[str] = set()
     runs = []
-    for run_id, created, status, run_type, scores_json, config_json in rows:
+    for run_id, created, status, run_type, scores_json, _config_json in rows:
         runs.append({"run_id": run_id, "created_at": created, "status": status, "run_type": run_type})
         if status != "completed":
             continue
         sc = _loads(scores_json)
         if run_type == "tool_eval":
+            excluded |= set(sc.get("excluded_scenarios") or ())  # upstream lists them only when non-empty
             cs = sc.get("category_scores") or []
             std = [c for c in cs if c.get("category") != HARD_CATEGORY]
             hard = [c for c in cs if c.get("category") == HARD_CATEGORY]
@@ -160,8 +169,13 @@ def summarize_db(db: Path) -> dict:
             continue
         last = dict(items[-1])
         last.pop("score", None)
+        if all("max" in i for i in items):  # every trial's points: deltas rescale each trial exactly
+            last["earneds"], last["maxes"] = [i["earned"] for i in items], [i["max"] for i in items]
+        comps = [i["completion_rate"] for i in items if i.get("completion_rate") is not None]
+        if comps:
+            last["completion_rate"] = min(comps)  # the worst trial: an exclusion in any trial shows
         suites[name] = {"score": round(mean(scores), 2), "scores": scores, "n": len(scores), **last}
-    return {
+    out = {
         "suites": suites,
         "categories": {
             k: {"label": v["label"], "percent": round(mean(v["percent"]), 2)}
@@ -171,6 +185,9 @@ def summarize_db(db: Path) -> dict:
         "scenarios": {k: round(mean(v), 3) for k, v in sorted(scen.items())},
         "runs": runs,
     }
+    if excluded:  # e.g. TC-45: the tool_choice=required probe failed — scored as zero on the full max
+        out["excluded"] = sorted(excluded)
+    return out
 
 
 # ── the run ─────────────────────────────────────────────────────────────────
@@ -204,9 +221,7 @@ def leg_label(rec: dict) -> str:
     if "MJOLNIR" not in cfg:
         return f"Mjolnir image · {BACKEND_LABELS.get(cfg, cfg)} · {kv}"
     extra = "".join(
-        f" + {name}"
-        for tag, name in (("LMH8", "fp8 lm_head"), ("VT32K", "32K draft vocab"), ("VT64K", "64K draft vocab"))
-        if tag in cfg.split("_")
+        f" + {name}" for tag, name in (("LMH8", "fp8 lm_head"), ("VT64K", "64K draft vocab")) if tag in cfg.split("_")
     )
     return f"Mjolnir · {kv}{extra}"
 
@@ -223,6 +238,7 @@ def run_accuracy(
     tool-eval-bench against it, and write ``accuracy.json``. Returns its path."""
     from mjolnir_cli import dockerctl
     from mjolnir_cli.config import config_field
+    from mjolnir_cli.platforms import detect, gpu_name
 
     tool = find_tool()
     run_id = time.strftime("%Y%m%d-%H%M%S")
@@ -232,6 +248,10 @@ def run_accuracy(
         "format": FORMAT,
         "run_id": run_id,
         "created": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+        "host": host_platform.node() or "unknown",
+        "gpu": gpu_name(),
+        # the detection-first rule: the record names the *detected* platform
+        "platform": detect().platform,
         "tool": {"repo": TOOL_REPO, "commit": TOOL_COMMIT, "command": tool},
         "args": list(passthrough) or list(DEFAULT_ARGS),
         "legs": [],
@@ -292,36 +312,132 @@ def load_runs(layout) -> list[dict]:
 
 
 def latest_legs(docs: list[dict]) -> list[dict]:
-    """One row per (model, image, config): every suite from the newest run
-    that measured it (an A/B run and a later GSM8K run merge into one row).
-    Order: first appearance."""
+    """One row per (platform, model, image side, config): every suite from the
+    newest run that measured it (an A/B run and a later GSM8K run merge into
+    one row). Order: first appearance. The platform comes from the run doc
+    (platform-less records are the repo's Thor history); a leg measured on
+    two platforms is two rows — its charts are per (platform, model). The
+    side (official image vs Mjolnir) keys the row, not the image tag, so a
+    re-measure on a newer image of the same side supersedes the older scores
+    in place (the row keeps its position and the newest image) instead of
+    duplicating a row the labels cannot tell apart."""
     rows: dict[tuple, dict] = {}
     for d in docs:
+        plat = platform_of(d)
         for leg in d.get("legs", []):
-            key = (leg.get("model"), leg.get("image"), leg.get("config"))
+            key = (plat, model_of(leg), _is_mjolnir(str(leg.get("image", ""))), leg.get("config"))
             row = rows.setdefault(
                 key,
                 {k: leg.get(k) for k in ("id", "model", "image", "config", "label", "kv_cache_dtype", "env")}
-                | {"suites": {}, "categories": {}, "scenarios": {}, "runs": {}},
+                | {"platform": plat, "suites": {}, "categories": {}, "scenarios": {}, "runs": {}},
             )
+            row["image"] = leg.get("image") or row.get("image")
             row["kv_cache_dtype"] = leg.get("kv_cache_dtype") or row.get("kv_cache_dtype")
             row["label"] = leg_label(row)  # recomputed: labels improve after a run is stored
             for name, suite in (leg.get("suites") or {}).items():
                 row["suites"][name] = suite
                 row["runs"][name] = {"run_id": d.get("run_id"), "args": d.get("args")}
+            if leg.get("excluded"):  # an exclusion in any run of the leg shows
+                row["excluded"] = sorted({*(row.get("excluded") or ()), *leg["excluded"]})
             if (leg.get("suites") or {}).keys() & {"tools", "hardmode"}:
                 row["categories"] = leg.get("categories") or row["categories"]
                 row["scenarios"] = leg.get("scenarios") or row["scenarios"]
-    return [r for r in rows.values() if r["suites"]]
+    groups = by_model(r for r in rows.values() if r["suites"])
+    if len(groups) > 1:  # rows of several models: each model's rows together, labels say which
+        for r in (r for g in groups.values() for r in g):
+            r["label"] = f"{model_short(r)} · {r['label']}"
+    return [r for g in groups.values() for r in g]
 
 
 def baseline(rows: list[dict]) -> dict | None:
-    """The reference row for deltas: native FlashInfer on the official image
-    when measured, else the first row."""
-    for r in rows:
-        if not _is_mjolnir(str(r.get("image", ""))) and r.get("config") == "NVFP4":
+    """The reference row for deltas among ONE model's rows: native FlashInfer
+    on the official image when measured, else the first official-image row,
+    else the first row. For mixed rows use :func:`baselines`."""
+    official = [r for r in rows if not _is_mjolnir(str(r.get("image", "")))]
+    for r in official:
+        if r.get("config") == "NVFP4":
             return r
-    return rows[0] if rows else None
+    return (official or rows or [None])[0]
+
+
+def baselines(rows: list[dict]) -> list[dict | None]:
+    """Each row's baseline, in row order: its own model's :func:`baseline`.
+    Rows of another model are never a baseline — their scores are not comparable."""
+    per_model = {m: baseline(group) for m, group in by_model(rows).items()}
+    return [per_model[model_of(r)] for r in rows]
+
+
+def _maxes(s: dict) -> list:
+    """Every trial's max for a suite (older records keep only the last trial's)."""
+    return list(s.get("maxes") or ([s["max"]] if s.get("max") else []))
+
+
+def _on_max(s: dict, m: float) -> float:
+    """The suite's score with every trial's points over ``m``: exact from the
+    per-trial points when recorded, else from the one run's points, else from
+    the mean % (an older averaged record, which keeps only the last max)."""
+    if s.get("earneds"):
+        return mean(e / m * 100 for e in s["earneds"])
+    if s.get("n", 1) == 1 and "earned" in s:
+        return s["earned"] / m * 100
+    return s["score"] * s["max"] / m
+
+
+def delta(r: dict, base: dict | None, name: str) -> tuple[float, bool] | None:
+    """Points ``r`` scored over ``base`` on suite ``name``, and whether an
+    excluded scenario was scored as zero (None when ``r`` is the baseline,
+    either lacks the suite, or the two measured different scenario sets).
+
+    tool-eval-bench drops a scenario it judged the endpoint unable to serve
+    (TC-45 when its ``tool_choice=required`` probe fails, a timeout, an error),
+    which shrinks that trial's max and lowers its completion rate, so its %
+    divides by less. Then every trial of both legs is scored on the largest
+    max, the excluded points counting as zero: an exclusion never raises a
+    delta. Different maxes at full completion mean different scenario sets
+    (another run's arguments), which no rescaling makes comparable."""
+    if base is None or r is base:
+        return None
+    v, b = r.get("suites", {}).get(name) or {}, base.get("suites", {}).get(name) or {}
+    if v.get("score") is None or b.get("score") is None:
+        return None
+    maxes = _maxes(v) + _maxes(b)
+    if len(set(maxes)) > 1:
+        if all(x.get("completion_rate", 100.0) >= 100.0 for x in (v, b)):
+            return None
+        m = max(maxes)
+        return _on_max(v, m) - _on_max(b, m), True
+    return v["score"] - b["score"], False
+
+
+def fmt_delta(d: tuple[float, bool] | None) -> str:
+    """' (+1.2)', ' (-1.4‡)' when an excluded scenario was scored as zero, '' for no delta."""
+    return f" ({d[0]:+.1f}{'‡' if d[1] else ''})" if d else ""
+
+
+def leg_excluded(r: dict, base: dict | None, names: list[str]) -> bool:
+    """Whether any of ``r``'s suite deltas scored an excluded scenario as zero."""
+    return any((d := delta(r, base, n)) and d[1] for n in names)
+
+
+def any_excluded(rows: list[dict], bases: list[dict | None], names: list[str]) -> bool:
+    return any(leg_excluded(r, b, names) for r, b in zip(rows, bases, strict=True))
+
+
+EXCLUDED_NOTE = (
+    "‡ = tool-eval-bench excluded a scenario on one of the two legs; the delta scores both on the full max, "
+    "the excluded points counting as zero"
+)
+EXCLUDED_NOTE_SHORT = "‡ = a scenario was excluded on one side, scored as zero"
+CATEGORY_EXCLUDED_NOTE = (
+    "‡ = this leg had a scenario excluded: its category % divides by the smaller max, so one category reads high"
+)
+
+
+def baseline_caption(bases: list[dict | None], item_sep: str = " · ") -> str:
+    """The distinct baseline labels, first-seen order, ' / '-joined ('' if none);
+    ``item_sep`` replaces the label's own ' · ' separator."""
+    labels = dict.fromkeys(b["label"].replace(" · ", item_sep) for b in bases if b is not None)
+    return " / ".join(labels)
 
 
 def suites_in(rows: list[dict]) -> list[str]: