patchworks 2.6.10__tar.gz → 2.6.12__tar.gz

{patchworks-2.6.10 → patchworks-2.6.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: patchworks
-Version: 2.6.10
+Version: 2.6.12
 Summary: Tiled processing of arbitrarily large images with globally consistent labels
 Project-URL: Homepage, https://github.com/imcf/patchworks
 Project-URL: Issues, https://github.com/imcf/patchworks/issues

{patchworks-2.6.10 → patchworks-2.6.12}/docs/examples/dog.md

@@ -102,6 +102,21 @@ decon_kwargs=dict(psf=psf, dxpsf=0.05, dzpsf=0.1, wavelength=525, ...)
     so edge tiles keep enough context (a plain intensity/threshold halo is
     too thin).
 
+!!! note "cudaDecon can return a smaller volume than it was given"
+    It rounds each axis down to an FFT-efficient length — e.g. a
+    `(32, 1084, 1084)` tile comes back `(32, 1080, 1080)`, because
+    `1080 = 2³·3³·5` while `1084 = 4·271` — and trims the excess off the
+    **high end**, leaving voxel `(0, 0, 0)` where it was. patchworks restores
+    the input shape before the DoG step (one label per input voxel is
+    required) anchored at that origin, and logs a WARNING with both shapes.
+
+    Anchoring matters: restoring it *centred* instead moves every voxel by
+    `excess // 2` — a 2 px y/x shift for the tile above, identical on every
+    tile. That is invisible on a cell tens of voxels wide and obvious on a
+    cilium a few voxels wide, which is how it was eventually caught. If the
+    logged difference is more than a few voxels, the PSF or the voxel sizes
+    are wrong.
+
 ## Growing the labels afterwards
 
 DoG spots/threads are often thin — grow each label by a few pixels with

{patchworks-2.6.10 → patchworks-2.6.12}/docs/guide/snakemake.md

@@ -112,15 +112,60 @@ sequential_labels: true        # renumber labels to a contiguous 1..N
     [Filtering by size after merge](merging.md#filtering-by-size-after-merge)
     for the equivalent direct-API call.
 
+!!! warning "`model:` names are version-specific — check which Cellpose you have"
+    Cellpose 3 and 4 have **disjoint** model names: v3 has `cyto3`,
+    `nuclei`, …; v4 replaced them all with the `cpsam` family (`cpsam`,
+    `cpsam_v2`, `cpdino`, …). Neither version *raises* on a name it doesn't
+    know — v4 logs a warning and quietly loads its default (`cpsam_v2`)
+    instead. A `cyto3` left in a config against a v4 install therefore
+    segments every tile with a model you did not choose, and the only
+    evidence is one line in the job log.
+
+    patchworks now rejects an unavailable name in `prepare`, on the cheap
+    CPU job, rather than letting it through. Check what you have with
+    `python -c "import cellpose; print(cellpose.version)"`, then either pick
+    a name that install offers, point `model:` at a custom-trained model's
+    path, or install the version you want — the workflow ships `cellpose3`
+    and `cellpose4` pixi environments (`pixi install -e cellpose3`) for
+    exactly this.
+
 !!! tip "3-D anisotropy is derived automatically"
     Cellpose's `do_3D` assumes isotropic voxels unless told otherwise —
     without an `anisotropy`, a real (anisotropic) dataset gets objects
-    fragmented or distorted across z. `segment` now derives it from
+    fragmented or distorted across z. `segment` derives it from
     `image.zarr`'s own calibration (`z` voxel size ÷ lateral voxel size)
     whenever `do_3D: true` and `cellpose.anisotropy` isn't set explicitly, so
     there's usually nothing to configure. Set `anisotropy:` yourself in the
     `cellpose:` block to override it.
 
+    Note it is not just a hint to the model: Cellpose *resizes* the tile to
+    `z × anisotropy` planes before the net runs. `tile_shape: "auto"`
+    budgets for that resized tile, so a 2.2× anisotropy buys a
+    correspondingly smaller tile rather than an out-of-memory job.
+
+!!! warning "The physically-correct anisotropy is not always the best one"
+    Upsampling z by the true ratio can produce **ring artifacts** and costs
+    runtime proportional to the ratio. A contributor on
+    [cellpose#1408](https://github.com/MouseLand/cellpose/pull/1408) reports
+    that `anisotropy: 1` avoids the rings entirely and is much faster, at
+    the cost of boundaries being off by a few pixels in the top/bottom
+    z-planes, where a cell's cross-section changes fastest — and recommends
+    pairing it with light z-only flow smoothing:
+
+    ```yaml
+    cellpose:
+      do_3D: true
+      anisotropy: 1          # overrides the derived value
+      flow3D_smooth: [1, 0, 0]   # z, y, x -- smooth z only
+    ```
+
+    This is a genuine trade-off, not a strictly better setting, and it is
+    worth testing both on your own data rather than taking either on faith.
+    Reach for it especially if 3-D results look ringed or fragmented along
+    z: that is the symptom this addresses. `flow3D_smooth` accepts a scalar
+    on older Cellpose and a `[z, y, x]` list from the version that merged
+    that PR onward.
+
 !!! tip "Tile size vs runtime"
     `tile_shape: "auto"` sizes each tile to your GPU's VRAM. Smaller tiles =
     more (faster) jobs; very large 3-D tiles are slow. Keep `do_3D: false` (2-D
@@ -444,10 +489,14 @@ results/image.zarr/labels/cyto_labels/
 
 !!! tip "The relate step's log"
     Unlike `prepare`/`segment`/`merge`, the relate step isn't a Snakemake
-    rule, so it doesn't get a `log:` directive for free. It writes its own
-    log to `<work_dir>/logs/relate.log` (override with `relate.py --log`),
-    the same tee-to-file-and-stdout behaviour as the other steps — check
-    there instead of scrolling back through `srun`'s live output.
+    rule, so it doesn't get a `log:` directive for free. Standalone (or
+    under plain `multi`), it writes to `<work_dir>/logs/relate.log`
+    (override with `relate.py --log`), the same tee-to-file-and-stdout
+    behaviour as the other steps. Under `multi-slurm`, where each pair is
+    its own concurrent job, `run_multi.py` points each one at its own file
+    instead — `<work_dir>/logs/relate/<a>_to_<b>.log` — so concurrent pairs
+    don't interleave into one log; check there instead of scrolling back
+    through `srun`'s live output.
 
 See [Relating labels across segmentations](label_relations.md) for what
 `label_relations()` returns and how to save it yourself — the cluster
@@ -498,15 +547,29 @@ abort the others; you get a per-config status and a non-zero exit.
 
 !!! tip "The relate step runs on the cluster too, under `multi-slurm`"
     `label_relations()` streams every chunk of two full-resolution label
-    volumes — real CPU/IO work, not orchestration. Under `multi-slurm` it is
-    submitted as its own `srun` job (`scripts/relate.py`) instead of running
-    in the driver process on the login node, the same fix already applied to
-    the occupancy map. Tune its allocation with `--relate-partition`,
-    `--relate-mem`, `--relate-cpus` and `--relate-time` (defaults: `scicore`,
-    `32G`, `8`, `180` minutes) — these are wide-margin guesses, not measured
-    numbers, so raise them for a very large or very object-dense pair. Under
-    plain `multi` (no `--profile`), it still runs locally, in-process, as
-    before.
+    volumes — real CPU/IO work, not orchestration. Under `multi-slurm`,
+    **each pair in `relations:` is submitted as its own `srun` job**
+    (`scripts/relate.py`) instead of running in the driver process on the
+    login node, the same fix already applied to the occupancy map. Tune the
+    allocation with `--relate-partition`, `--relate-mem`, `--relate-cpus`
+    and `--relate-time` (defaults: `scicore`, `32G`, `8`, `180` minutes,
+    **per relation**) — these are wide-margin guesses, not measured numbers,
+    so raise them for a very large or very object-dense pair; a pair needing
+    a chunk-layout rechunk first (see below) is the usual reason one runs
+    long. Set `--relate-qos` if your account's default QOS for the partition
+    caps the wall time below `--relate-time` — `srun` fails immediately with
+    `QOSMaxWallDurationPerJobLimit` when that happens; `sacctmgr -p show
+    assoc user=$USER` and `sacctmgr -p show qos` list what's available and
+    each one's `MaxWall`. Under plain `multi` (no `--profile`), relations
+    still run locally, in-process, one after another, as before.
+
+    Because every pair gets its own job, one running long no longer starves
+    the others out of a shared time budget, and a pair that gets killed no
+    longer takes an already-finished sibling's workbook down with it.
+    `relate.py` also skips a pair whose `.xlsx` is already newer than both
+    labels' merge marker, so **re-running the exact same `multi-slurm`
+    command only recomputes what's still missing or stale** — delete a
+    specific `.xlsx` yourself to force just that one to recompute.
 
 !!! tip "After a killed run"
     Snakemake only releases its lock on a clean exit, so a run that was killed

{patchworks-2.6.10 → patchworks-2.6.12}/src/patchworks/_chunks.py

@@ -380,6 +380,7 @@ def auto_tile_shape_cellpose(
     model_memory_bytes: int = 2 * 1024**3,
     cellpose_memory_factor: int = 20,
     n_channels: int = 1,
+    anisotropy: float | None = None,
     verbose: bool = False,
 ) -> tuple[int, ...]:
     """Cellpose-optimised tile shape.
@@ -426,6 +427,13 @@ def auto_tile_shape_cellpose(
         Channels each tile carries (default 1). Above 1 the per-voxel cost
         scales with it, so the tile shrinks accordingly -- e.g. the workflow's
         ``nuclei_channel`` hands Cellpose a cyto+nuclei pair.
+    anisotropy:
+        The ``anisotropy`` Cellpose will be given (``do_3D`` only). It is not
+        merely a hint to the model: Cellpose resizes the tile to
+        ``z * anisotropy`` planes before the net runs, so a 2.2x anisotropy
+        costs 2.2x the z it was handed. Budgeting against the unscaled ``z``
+        under-counts by exactly that factor. ``None`` (the default) assumes
+        isotropic, i.e. no resize.
     verbose:
         Log the chosen shape and memory estimates.
 
@@ -477,7 +485,12 @@ def auto_tile_shape_cellpose(
             chunk_spatial = [1, min(y, tile_side), min(x, tile_side)]
     else:
         z, y, x = shape[-3], shape[-2], shape[-1]
-        max_pixels_per_slice = max(1, (max_raw_bytes // 3) // (z * itemsize))
+        # Cellpose resizes z by `anisotropy` before the net runs, so the tile
+        # it actually holds is that much deeper than the one handed to it.
+        effective_z = z * max(1.0, anisotropy or 1.0)
+        max_pixels_per_slice = max(
+            1, int((max_raw_bytes // 3) // (effective_z * itemsize))
+        )
         tile_side = max(min_tile, int(max_pixels_per_slice**0.5))
         chunk_spatial = [z, min(y, tile_side), min(x, tile_side)]
 

{patchworks-2.6.10 → patchworks-2.6.12}/src/patchworks/plugins/cellpose.py

@@ -253,6 +253,26 @@ def _make_config(
     }
 
 
+def available_models() -> list[str]:
+    """Pretrained model names the installed Cellpose actually accepts.
+
+    The name set changed completely between major versions -- v3 ships
+    ``cyto3``/``nuclei``/… , v4 replaced them with ``cpsam``-family names --
+    and neither version raises on an unknown one. v4 falls back to its
+    default model with only a log line, so a v3 name in a v4 environment
+    segments everything with the wrong model and nothing fails. Checking the
+    name against this list turns that into a config error instead.
+
+    Returns
+    -------
+    list of str
+        Model names, or an empty list when Cellpose isn't installed or
+        doesn't publish them (in which case no name can be rejected).
+    """
+    names = getattr(_cellpose_models, "MODEL_NAMES", None)
+    return list(names) if names else []
+
+
 def _get_model(cellpose_dict: dict[str, Any]) -> Any:
     """Return a worker-local cached Cellpose model.
 
@@ -272,8 +292,14 @@ def _get_model(cellpose_dict: dict[str, Any]) -> Any:
         gpu = cellpose_dict.get("gpu", False)
         model_type = cellpose_dict["model"]
         if _CELLPOSE_V4:
+            # v4 renamed this: `model_type=` is accepted but explicitly
+            # ignored ("not used in v4.0.1+"), leaving pretrained_model at
+            # its "cpsam_v2" default -- so passing the configured name there
+            # silently segmented *every* config with the same default model,
+            # whatever `model:` said, with only a logger warning to show for
+            # it. `available_models()` rejects an unusable name up front.
             _model_cache[key] = _cellpose_models.CellposeModel(
-                model_type=model_type, gpu=gpu
+                pretrained_model=model_type, gpu=gpu
             )
         else:
             _model_cache[key] = _cellpose_models.Cellpose(

{patchworks-2.6.10 → patchworks-2.6.12}/src/patchworks/plugins/dog.py

@@ -239,18 +239,23 @@ def _run(block: np.ndarray, dog_dict: dict[str, Any]) -> np.ndarray:
 
 
 def _restore_shape(arr: np.ndarray, shape: tuple[int, ...]) -> np.ndarray:
-    """Centre *arr* back into an array of *shape*, cropping or edge-padding.
+    """Restore *arr* to *shape*, anchored at the origin, cropping or padding.
 
     Deconvolution must not change the field of view: patchworks writes the
     result into a destination slice derived from the tile's geometry, so one
     label per input voxel is required.
 
-    Centring is the right correction for a symmetric crop, which is what
-    apodisation produces. The discrepancies observed are small (a voxel in z,
-    a few in x/y) and land inside the halo, which is discarded anyway -- so
-    the labels that survive the trim are unaffected. It is logged at WARNING
-    with the exact shapes so a larger, non-symmetric crop cannot pass
-    silently.
+    The alignment is **origin-anchored**, not centred: cudaDecon rounds each
+    axis down to an FFT-efficient length (e.g. 1084 -> 1080, since
+    1080 = 2**3 * 3**3 * 5 while 1084 = 4 * 271) and trims the excess off the
+    high end, leaving voxel (0, 0, 0) where it was. Re-centring content that
+    was never centred shifts every voxel by ``excess // 2`` -- measured at 2
+    px in y and x on a real (32, 1084, 1084) tile, in the same direction on
+    every tile. That is invisible on a cell tens of voxels across and glaring
+    on a cilium a few voxels across, which is exactly how it was found.
+
+    A mismatch is still logged at WARNING with the exact shapes, since a
+    large one means the PSF or voxel sizes are wrong.
 
     Parameters
     ----------
@@ -271,22 +276,22 @@ def _restore_shape(arr: np.ndarray, shape: tuple[int, ...]) -> np.ndarray:
     (14, 1024)
     """
     logger.warning(
-        "deconvolution returned %s for a %s input; re-centring to the input "
-        "shape. patchworks needs one label per input voxel. A large or "
-        "asymmetric difference here would shift labels -- check the PSF and "
-        "voxel sizes if this is more than a few voxels.",
+        "deconvolution returned %s for a %s input; restoring the input shape "
+        "from the origin. patchworks needs one label per input voxel. A large "
+        "difference here means the PSF or voxel sizes are wrong -- check them "
+        "if this is more than a few voxels.",
         arr.shape,
         shape,
     )
     # Crop first, so an axis that grew is handled before padding the rest.
+    # Both keep voxel 0 where it is: cudaDecon trims off the high end (see the
+    # docstring), so the low corner is the one landmark known to be unmoved.
     crop = tuple(
-        slice((a - s) // 2, (a - s) // 2 + s) if a > s else slice(None)
-        for a, s in zip(arr.shape, shape)
+        slice(0, s) if a > s else slice(None) for a, s in zip(arr.shape, shape)
     )
     arr = arr[crop]
     pad = tuple(
-        ((s - a) // 2, s - a - (s - a) // 2) if a < s else (0, 0)
-        for a, s in zip(arr.shape, shape)
+        (0, s - a) if a < s else (0, 0) for a, s in zip(arr.shape, shape)
     )
     if any(lo or hi for lo, hi in pad):
         arr = np.pad(arr, pad, mode="edge")

{patchworks-2.6.10 → patchworks-2.6.12}/tests/test_allocation.py

@@ -117,3 +117,32 @@ def test_gpu_tile_sizing_is_bounded_by_the_host_allocation(monkeypatch):
         "the 1 GiB host grant must shrink the tile below what the same "
         "24 GiB GPU would otherwise allow"
     )
+
+
+def test_auto_tile_shape_cellpose_budgets_for_the_anisotropy_resize():
+    """Cellpose resizes a do_3D tile to z * anisotropy planes before the net
+
+    runs, so budgeting against the unscaled z hands the GPU a tile that is
+    `anisotropy` times bigger than the estimate. Auto-deriving anisotropy
+    turned that from dormant into live, so the sizer has to know about it.
+    """
+    from patchworks import auto_tile_shape_cellpose
+
+    kwargs = dict(
+        shape=(64, 4096, 4096),
+        dtype="uint16",
+        do_3D=True,
+        use_gpu=True,
+        gpu_memory=8 * 1024**3,
+        available_memory=64 * 1024**3,
+    )
+    isotropic = auto_tile_shape_cellpose(**kwargs)
+    anisotropic = auto_tile_shape_cellpose(**kwargs, anisotropy=2.215)
+
+    # z is pinned to the full extent either way; the cost is paid in y/x.
+    assert anisotropic[0] == isotropic[0]
+    assert anisotropic[1] < isotropic[1]
+    assert anisotropic[2] < isotropic[2]
+    # An anisotropy at or below 1 cannot *grow* the budget.
+    assert auto_tile_shape_cellpose(**kwargs, anisotropy=1.0) == isotropic
+    assert auto_tile_shape_cellpose(**kwargs, anisotropy=0.5) == isotropic

{patchworks-2.6.10 → patchworks-2.6.12}/tests/test_cellpose.py

@@ -47,3 +47,27 @@ def test_cellpose_fn_declares_a_voxel_size_parameter():
     # model.eval() argument, not a patchworks-specific one) -- only the raw
     # calibration is a named parameter.
     assert "anisotropy" not in params
+
+
+def test_available_models_is_a_list():
+    """Empty when Cellpose isn't installed -- then no name can be rejected."""
+    from patchworks.plugins.cellpose import available_models
+
+    assert isinstance(available_models(), list)
+
+
+def test_v4_gets_the_model_name_as_pretrained_model():
+    """v4 accepts `model_type=` and then ignores it ("not used in v4.0.1+"),
+
+    leaving pretrained_model at its default -- so passing the configured name
+    there segmented every config with the same default model whatever
+    `model:` said. The name has to reach `pretrained_model=` on v4.
+    """
+    import inspect
+
+    from patchworks.plugins import cellpose as cp
+
+    src = inspect.getsource(cp._get_model)
+    v4_branch = src.split("if _CELLPOSE_V4:")[1].split("else:")[0]
+    assert "pretrained_model=model_type" in v4_branch
+    assert "model_type=model_type" not in v4_branch

{patchworks-2.6.10 → patchworks-2.6.12}/tests/test_dog.py

@@ -109,20 +109,32 @@ def test_explicit_decon_kwargs_win_over_the_calibration(monkeypatch):
     assert captured["dzpsf"] == 0.2
 
 
-def test_restore_shape_recentres_a_cropped_decon():
+def test_restore_shape_anchors_a_cropped_decon_at_the_origin():
     """cudaDecon can hand back a smaller volume than it was given.
 
-    Observed on a real edge tile: (14, 1024, 1024) in, (13, 1020, 1020) out.
-    patchworks needs one label per input voxel, so the field of view has to be
-    restored before the DoG step.
+    Observed on a real tile: (32, 1084, 1084) in, (32, 1080, 1080) out --
+    each axis rounded down to an FFT-efficient length, with the excess taken
+    off the high end. Restoring it *centred* (what this used to do) moved
+    every voxel by excess // 2, measured as a 2 px y/x shift on real data:
+    invisible on a cell, glaring on a cilium a few voxels across.
     """
     from patchworks.plugins.dog import _restore_shape
 
     arr = np.arange(13 * 1020 * 1020, dtype="float32").reshape(13, 1020, 1020)
     out = _restore_shape(arr, (14, 1024, 1024))
     assert out.shape == (14, 1024, 1024)
-    # The original content is preserved, centred, not resampled.
-    assert np.array_equal(out[0:13, 2:1022, 2:1022], arr)
+    # Content keeps its original indices -- voxel 0 stays voxel 0.
+    assert np.array_equal(out[0:13, 0:1020, 0:1020], arr)
+
+
+def test_restore_shape_crops_from_the_high_end():
+    """The mirror case: an axis that came back too long keeps its low corner."""
+    from patchworks.plugins.dog import _restore_shape
+
+    arr = np.arange(6 * 12, dtype="float32").reshape(6, 12)
+    out = _restore_shape(arr, (4, 8))
+    assert out.shape == (4, 8)
+    assert np.array_equal(out, arr[0:4, 0:8])
 
 
 def test_restore_shape_handles_growth_and_exact_fit():

{patchworks-2.6.10 → patchworks-2.6.12}/tests/test_pw.py

@@ -121,3 +121,44 @@ def test_validate_config_rejects_max_volume_at_or_below_min_volume():
         validate_config(
             {"method": "threshold", "min_volume": 500.0, "max_volume": 5.0}
         )
+
+
+def test_validate_config_rejects_a_model_the_install_does_not_have(
+    monkeypatch,
+):
+    """Cellpose does not raise on an unknown model name -- it logs and loads
+
+    its default. A v3 name against a v4 install therefore segments every
+    tile with a model nobody chose, silently. That has to fail in prepare.
+    """
+    import pytest
+    from _pw import validate_config
+
+    import patchworks.plugins.cellpose as cp
+
+    monkeypatch.setattr(cp, "available_models", lambda: ["cpsam", "cpsam_v2"])
+    with pytest.raises(ValueError, match="cyto3"):
+        validate_config({"method": "cellpose", "cellpose": {"model": "cyto3"}})
+
+
+def test_validate_config_accepts_a_model_the_install_has(monkeypatch):
+    from _pw import validate_config
+
+    import patchworks.plugins.cellpose as cp
+
+    monkeypatch.setattr(cp, "available_models", lambda: ["cpsam", "cpsam_v2"])
+    validate_config({"method": "cellpose", "cellpose": {"model": "cpsam"}})
+
+
+def test_validate_config_leaves_a_custom_model_path_alone(
+    monkeypatch, tmp_path
+):
+    """A path is a custom-trained model; no name list can vouch for it."""
+    from _pw import validate_config
+
+    import patchworks.plugins.cellpose as cp
+
+    monkeypatch.setattr(cp, "available_models", lambda: ["cpsam"])
+    custom = tmp_path / "my_model.pth"
+    custom.write_text("")
+    validate_config({"method": "cellpose", "cellpose": {"model": str(custom)}})

{patchworks-2.6.10 → patchworks-2.6.12}/tests/test_run_multi.py

@@ -1,7 +1,10 @@
 """Tests for the multi-config driver's SLURM-facing behaviour."""
 
+import json
+import os
 import re
 import sys
+import time
 from pathlib import Path
 
 sys.path.insert(
@@ -15,6 +18,7 @@ import yaml  # noqa: E402
 
 from run_multi import (  # noqa: E402
     _CONVERT_KEYS,
+    _relate_cmd,
     _snakemake_cmd,
     _validate_configs,
     slurm_jobname_prefix,
@@ -176,6 +180,85 @@ def test_relate_is_submitted_via_slurm_under_profile():
     assert "label_relations(" not in src
 
 
+def _relate_kwargs(**overrides):
+    kwargs = dict(
+        work_dir="/w",
+        image_store="/w/image.zarr",
+        workflow_dir=Path("/workflow"),
+        relate_partition="scicore",
+        relate_mem="32G",
+        relate_cpus=8,
+        relate_time=180,
+        relate_qos=None,
+    )
+    kwargs.update(overrides)
+    return kwargs
+
+
+def test_relate_cmd_is_one_job_per_relation_pair():
+    """A killed shared job used to lose every relation still queued behind
+
+    the one that was running -- one srun per pair means a slow or failing
+    pair can no longer starve, or take down, its siblings' time budget.
+    """
+    relations = [
+        {"a": "nuclei_labels", "b": "cyto_labels", "output": "n2c.xlsx"},
+        {"a": "cilia_labels", "b": "cyto_labels", "output": "c2c.xlsx"},
+    ]
+    cmds = [_relate_cmd(rel, **_relate_kwargs()) for rel in relations]
+
+    assert len(cmds) == 2
+    for cmd, rel in zip(cmds, relations):
+        assert cmd[0] == "srun"
+        assert "--relations" in cmd
+        # Each job's payload is *only* its own pair, not the whole list.
+        payload = json.loads(cmd[cmd.index("--relations") + 1])
+        assert payload == [rel]
+
+
+def test_relate_cmd_job_name_identifies_the_pair():
+    cmd = _relate_cmd(
+        {"a": "cilia_labels", "b": "cyto_labels"}, **_relate_kwargs()
+    )
+    name = cmd[cmd.index("--job-name") + 1]
+    assert _EXECUTOR_RULE.match(name)
+    assert "cilia_labels" in name
+    assert "cyto_labels" in name
+
+
+def test_relate_cmd_gives_each_pair_its_own_log():
+    """Concurrent per-pair jobs sharing one relate.log would interleave --
+
+    each pair's --log must be a distinct file, or the whole point of
+    splitting the log the way segment/<batch>.log already does is lost.
+    """
+    cmds = [
+        _relate_cmd(rel, **_relate_kwargs())
+        for rel in (
+            {"a": "nuclei_labels", "b": "cyto_labels"},
+            {"a": "cilia_labels", "b": "cyto_labels"},
+        )
+    ]
+    logs = [cmd[cmd.index("--log") + 1] for cmd in cmds]
+    assert len(set(logs)) == 2
+    assert all(log.startswith("/w/logs/relate/") for log in logs)
+
+
+def test_relate_cmd_omits_qos_by_default():
+    cmd = _relate_cmd({"a": "a", "b": "b"}, **_relate_kwargs())
+    assert "--qos" not in cmd
+
+
+def test_relate_cmd_passes_qos_when_set():
+    """A default QOS whose MaxWall is shorter than --relate-time is exactly
+
+    what killed a real run (QOSMaxWallDurationPerJobLimit) -- --relate-qos
+    lets a longer one be requested explicitly instead of guessed at.
+    """
+    cmd = _relate_cmd({"a": "a", "b": "b"}, **_relate_kwargs(relate_qos="1day"))
+    assert cmd[cmd.index("--qos") + 1] == "1day"
+
+
 def test_relate_script_has_the_real_bookkeeping():
     """relate.py must be the actual implementation, not a stub.
 
@@ -285,6 +368,88 @@ def test_relate_rechunks_mismatched_label_arrays(tmp_path):
     assert rows[2] == (None, 0, 0)  # label 2 touches nothing in b
 
 
+def test_relation_up_to_date_missing_output_is_false(tmp_path):
+    from relate import _relation_up_to_date
+
+    assert not _relation_up_to_date(
+        str(tmp_path), "a", "b", tmp_path / "nope.xlsx"
+    )
+
+
+def test_relation_up_to_date_missing_marker_is_false(tmp_path):
+    """No labels.done for a label means its state can't be judged -- treat
+
+    that as "recompute", not as "trust the existing workbook".
+    """
+    from relate import _relation_up_to_date
+
+    out = tmp_path / "rel.xlsx"
+    out.write_text("x")
+    assert not _relation_up_to_date(str(tmp_path), "a", "b", out)
+
+
+def test_relation_up_to_date_true_only_when_newer_than_both_markers(
+    tmp_path,
+):
+    from relate import _relation_up_to_date
+
+    (tmp_path / "a").mkdir()
+    (tmp_path / "b").mkdir()
+    (tmp_path / "a" / "labels.done").touch()
+    (tmp_path / "b" / "labels.done").touch()
+    out = tmp_path / "rel.xlsx"
+    out.write_text("x")
+
+    now = time.time()
+    os.utime(tmp_path / "a" / "labels.done", (now, now))
+    os.utime(tmp_path / "b" / "labels.done", (now, now))
+
+    # older than both markers -> stale
+    os.utime(out, (now - 10, now - 10))
+    assert not _relation_up_to_date(str(tmp_path), "a", "b", out)
+
+    # newer than both markers -> up to date
+    os.utime(out, (now + 10, now + 10))
+    assert _relation_up_to_date(str(tmp_path), "a", "b", out)
+
+    # b re-merged after the workbook was written -> stale again
+    os.utime(tmp_path / "b" / "labels.done", (now + 20, now + 20))
+    assert not _relation_up_to_date(str(tmp_path), "a", "b", out)
+
+
+def test_relate_skips_a_relation_whose_workbook_is_up_to_date(tmp_path):
+    """A retry must not recompute what already finished -- only what a
+
+    shared, now-split-per-pair job left missing after a partial failure.
+    Proven by planting sentinel content no real relation would produce: if
+    run_relations() recomputed anyway, the sentinel would be gone.
+    """
+    from relate import run_relations
+
+    work_dir = tmp_path / "work"
+    work_dir.mkdir()
+    for name in ("a_labels", "b_labels"):
+        (work_dir / name).mkdir()
+        (work_dir / name / "labels.done").touch()
+
+    out_path = work_dir / "rel.xlsx"
+    sentinel = openpyxl.Workbook()
+    sentinel.active.append(["sentinel"])
+    sentinel.save(out_path)
+    os.utime(out_path, (time.time() + 60, time.time() + 60))
+
+    # No image_store/labels at all -- if this weren't skipped, run_relations
+    # would raise trying to open them, not just produce the wrong content.
+    run_relations(
+        str(work_dir),
+        str(tmp_path / "image.zarr"),
+        [{"a": "a_labels", "b": "b_labels", "output": "rel.xlsx"}],
+    )
+
+    wb = openpyxl.load_workbook(out_path)
+    assert wb.active["A1"].value == "sentinel"
+
+
 def test_mixed_nuclei_channel_auto_passes_validation():
     """A channel-count mismatch under `tile_shape: "auto"` is no longer