patchworks 2.6.5__tar.gz → 2.6.7__tar.gz

{patchworks-2.6.5 → patchworks-2.6.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: patchworks
-Version: 2.6.5
+Version: 2.6.7
 Summary: Tiled processing of arbitrarily large images with globally consistent labels
 Project-URL: Homepage, https://github.com/imcf/patchworks
 Project-URL: Issues, https://github.com/imcf/patchworks/issues

{patchworks-2.6.5 → patchworks-2.6.7}/docs/guide/label_relations.md

@@ -6,7 +6,21 @@ the other — by streaming both arrays chunk by chunk, so it scales to
 hundreds of thousands of objects without loading anything fully into RAM.
 
 Both label arrays must share the exact same chunk layout — same
-`tile_shape`/pyramid `level` when they were produced.
+`tile_shape`/pyramid `level` when they were produced. `label_relations()`
+raises rather than silently doing something slower if they don't; rechunk
+one side to match first (same shape, different chunking is a normal dask
+op — some extra I/O reading across misaligned source chunks, not a
+correctness issue):
+
+```python
+if nuclei.chunks != cells.chunks:
+    cells = cells.rechunk(nuclei.chunks)
+```
+
+(The cluster workflow's `run_multi.py`/`relate.py` does this automatically
+— see below — so two configs segmented at different `tile_shape` still
+relate correctly; you only need to do it by hand when calling
+`label_relations()` directly.)
 
 ```python
 import dask.array as da

{patchworks-2.6.5 → patchworks-2.6.7}/docs/guide/snakemake.md

@@ -398,20 +398,29 @@ results/image.zarr/labels/nuclei_labels/
 results/image.zarr/labels/cyto_labels/
 ```
 
-!!! tip "Keep `level` identical across configs; `tile_shape` usually takes care of itself"
+!!! tip "Keep `level` identical across configs; `tile_shape` doesn't have to match anymore"
     Different segmentations of the same image can use different `channel`
     and `cellpose:` settings freely, but keep `level` the same across
-    configs — the label arrays then share the exact same chunk layout, which
-    [`label_relations()`](label_relations.md) requires.
-
-    `tile_shape` is a little more forgiving under `run_multi.py`: leaving it
-    on `"auto"` (the default) would normally size a `nuclei_channel` config's
-    tile smaller than a single-channel one (the sizer charges per channel),
-    breaking that same requirement — `run_multi.py` detects this and pins
-    every config to the smallest computed tile automatically, so mixed
-    single-/two-channel configs under `"auto"` just work. Setting an
-    explicit `tile_shape` yourself still requires it to be identical across
-    configs, same as before.
+    configs so the label arrays cover the same voxel grid.
+
+    `tile_shape` itself no longer has to match. Two configs naturally
+    produce different on-disk chunk layouts when `tile_shape: "auto"`
+    charges a `nuclei_channel` config for a smaller tile than a
+    single-channel one, or when one config was segmented before the other's
+    settings changed, or a cheaper method (e.g. a DoG detector) sized its
+    own tile differently — `relate.py` detects a chunk mismatch per pair and
+    rechunks the finer side to match before relating, so this is no longer
+    something you need to plan around. (`run_multi.py` *also* auto-pins
+    every config under `"auto"` to one shared, smallest-computed tile
+    up front — mainly useful to keep segmentation itself GPU-efficient
+    across configs, not required for the relate step to work.)
+
+!!! tip "The relate step's log"
+    Unlike `prepare`/`segment`/`merge`, the relate step isn't a Snakemake
+    rule, so it doesn't get a `log:` directive for free. It writes its own
+    log to `<work_dir>/logs/relate.log` (override with `relate.py --log`),
+    the same tee-to-file-and-stdout behaviour as the other steps — check
+    there instead of scrolling back through `srun`'s live output.
 
 See [Relating labels across segmentations](label_relations.md) for what
 `label_relations()` returns and how to save it yourself — the cluster

{patchworks-2.6.5 → patchworks-2.6.7}/tests/test_run_multi.py

@@ -8,6 +8,8 @@ sys.path.insert(
     0, str(Path(__file__).resolve().parents[1] / "workflow" / "scripts")
 )
 
+import numpy as np  # noqa: E402
+import openpyxl  # noqa: E402
 import pytest  # noqa: E402
 import yaml  # noqa: E402
 
@@ -186,6 +188,77 @@ def test_relate_script_has_the_real_bookkeeping():
     assert "openpyxl" in src
 
 
+def test_relate_writes_its_own_log():
+    """relate.py runs via srun, not a Snakemake rule -- nothing else wires up
+
+    its logging (see prepare/segment/merge's `log:` directives), so main()
+    has to call start_log() itself or its output only ever streams to
+    whatever invoked srun and is gone once that terminal scrolls past it.
+    """
+    src = (_workflow_dir() / "scripts" / "relate.py").read_text()
+    assert "from _pw import start_log" in src
+    assert "start_log(" in src
+    assert '"logs" / "relate.log"' in src
+
+
+def test_relate_rechunks_mismatched_label_arrays(tmp_path):
+    """A chunk-layout mismatch must be rechunked away, not require a re-run.
+
+    Two configs are free to have segmented at different tile_shape (one
+    published before the other's config changed, or a cheaper method sized
+    its own tile differently) -- label_relations() itself refuses mismatched
+    chunks by design, but that only means the caller has to rechunk one side
+    first, not that the whole segmentation needs redoing.
+    """
+    import zarr
+
+    from relate import run_relations
+
+    image_store = str(tmp_path / "image.zarr")
+
+    # a: labels 1 and 2, split at x=5. b: a single label 10 covering all of
+    # a's label 1 and none of label 2 -- built with a *different* chunking.
+    a_data = np.zeros((1, 10), dtype=np.int32)
+    a_data[0, :5] = 1
+    a_data[0, 5:] = 2
+    b_data = np.zeros((1, 10), dtype=np.int32)
+    b_data[0, :5] = 10
+
+    root = zarr.open_group(image_store, mode="w")
+    labels = root.require_group("labels")
+    a_grp = labels.require_group("nuclei_labels")
+    a_arr = a_grp.create_array(
+        name="0", shape=a_data.shape, chunks=(1, 2), dtype=np.int32
+    )
+    a_arr[:] = a_data
+    a_grp.attrs["sequential_labels"] = True
+    a_grp.attrs["n_objects"] = 2
+
+    b_grp = labels.require_group("cyto_labels")
+    b_arr = b_grp.create_array(
+        name="0", shape=b_data.shape, chunks=(1, 5), dtype=np.int32
+    )
+    b_arr[:] = b_data
+    b_grp.attrs["sequential_labels"] = True
+    b_grp.attrs["n_objects"] = 1
+
+    out_dir = tmp_path / "work"
+    out_dir.mkdir()
+    run_relations(
+        str(out_dir),
+        image_store,
+        [{"a": "nuclei_labels", "b": "cyto_labels", "output": "rel.xlsx"}],
+    )
+
+    wb = openpyxl.load_workbook(out_dir / "rel.xlsx")
+    rows = {
+        row[0]: (row[1], row[2], row[3])
+        for row in wb["nuclei_labels"].iter_rows(min_row=2, values_only=True)
+    }
+    assert rows[1] == (10, 5, 1.0)  # label 1 fully inside b's label 10
+    assert rows[2] == (None, 0, 0)  # label 2 touches nothing in b
+
+
 def test_mixed_nuclei_channel_auto_passes_validation():
     """A channel-count mismatch under `tile_shape: "auto"` is no longer
 

{patchworks-2.6.5 → patchworks-2.6.7}/workflow/scripts/relate.py

@@ -10,15 +10,27 @@ e.g. under srun):
     python scripts/relate.py --work-dir /path/to/work_dir \
         --image-store /path/to/work_dir/image.zarr \
         --relations '[{"a": "nuclei_labels", "b": "cyto_labels", "output": "nuclei_to_cyto.xlsx"}]'
+
+Unlike prepare/segment/merge, this doesn't run as a Snakemake rule, so nothing
+wires up its own logs/<rule>/*.log by default -- srun just streams output to
+whatever invoked it. main() calls start_log() itself instead, writing to
+<work_dir>/logs/relate.log (override with --log), the same tee-to-file-and-
+stdout behaviour the Snakemake-driven scripts get via _pw.start_log.
 """
 
 from __future__ import annotations
 
 import argparse
 import json
+import math
 from pathlib import Path
 
 
+def _num_chunks(arr: "da.Array") -> int:  # noqa: F821 - dask imported lazily by callers
+    """Total chunk count of a dask array, for picking the coarser side to rechunk to."""
+    return math.prod(len(c) for c in arr.chunks)
+
+
 def _label_ids(image_store: str, name: str) -> list[int]:
     """Ids present in a label image, without scanning the volume.
 
@@ -72,6 +84,35 @@ def run_relations(
         print(f"[relate] relating {a_name} -> {b_name} …", flush=True)
         a = da.from_zarr(image_store, component=f"labels/{a_name}/0")
         b = da.from_zarr(image_store, component=f"labels/{b_name}/0")
+
+        # label_relations() requires matching chunk layouts (it walks both
+        # arrays block-by-block at the same index) but two configs are free
+        # to have segmented at different tile_shape -- e.g. one already
+        # published before the other's config changed, or a cheaper method
+        # naturally sized its tile differently. Same shape, different
+        # chunking is a normal dask op (extra I/O reading across misaligned
+        # source chunks, not a correctness issue), so rechunk the finer side
+        # to the coarser one here rather than require identical tile_shape
+        # across every config up front.
+        if a.chunks != b.chunks:
+            a_n, b_n = _num_chunks(a), _num_chunks(b)
+            if a_n <= b_n:
+                print(
+                    f"[relate] {a_name} chunks {a.chunks} != {b_name} "
+                    f"chunks {b.chunks}; rechunking {b_name} to match "
+                    f"{a_name} (fewer chunks)",
+                    flush=True,
+                )
+                b = b.rechunk(a.chunks)
+            else:
+                print(
+                    f"[relate] {a_name} chunks {a.chunks} != {b_name} "
+                    f"chunks {b.chunks}; rechunking {a_name} to match "
+                    f"{b_name} (fewer chunks)",
+                    flush=True,
+                )
+                a = a.rechunk(b.chunks)
+
         table = label_relations(a, b)
 
         # label_relations() only returns a-objects that touch a b-object.
@@ -141,7 +182,19 @@ def main() -> None:
             "relations:"
         ),
     )
+    parser.add_argument(
+        "--log",
+        default=None,
+        help="log file path (default: <work-dir>/logs/relate.log)",
+    )
     args = parser.parse_args()
+
+    from _pw import start_log
+
+    log_path = args.log or str(Path(args.work_dir) / "logs" / "relate.log")
+    start_log(log_path)
+    print(f"[relate] logging to {log_path}", flush=True)
+
     run_relations(args.work_dir, args.image_store, json.loads(args.relations))