patchworks 1.3.4__tar.gz → 1.4.1__tar.gz

{patchworks-1.3.4 → patchworks-1.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: patchworks
-Version: 1.3.4
+Version: 1.4.1
 Summary: Tiled processing of arbitrarily large images with globally consistent labels
 Project-URL: Homepage, https://github.com/imcf/patchworks
 Project-URL: Issues, https://github.com/imcf/patchworks/issues
@@ -143,6 +143,8 @@ pip install "patchworks[all]"       # Everything, incl. the napari viewer
 > `[imaris]` adds native `.ims` support (HDF5, no JVM). Physical pixel
 > calibration is read from the input and written into the OME-ZARR.
 
+<!-- -->
+
 > **`cupy` is never installed automatically**, unlike Cellpose's GPU support
 > (which comes for free via PyTorch's self-contained CUDA wheels). Any
 > `use_gpu=True`/`dilate_gpu: true` option (the `dog` plugin, `dilate_labels`)

{patchworks-1.3.4 → patchworks-1.4.1}/README.md

@@ -53,6 +53,8 @@ pip install "patchworks[all]"       # Everything, incl. the napari viewer
 > `[imaris]` adds native `.ims` support (HDF5, no JVM). Physical pixel
 > calibration is read from the input and written into the OME-ZARR.
 
+<!-- -->
+
 > **`cupy` is never installed automatically**, unlike Cellpose's GPU support
 > (which comes for free via PyTorch's self-contained CUDA wheels). Any
 > `use_gpu=True`/`dilate_gpu: true` option (the `dog` plugin, `dilate_labels`)

{patchworks-1.3.4 → patchworks-1.4.1}/docs/examples/dog.md

@@ -6,6 +6,9 @@ label the connected components. CPU (scipy) by default, GPU (cupy) optional.
 Optionally deconvolve each tile first with
 [pycudadecon](https://github.com/tlambert03/pycudadecon).
 
+> Cilia DoG + deconvolution approach courtesy of
+> [angelo-angonezi](https://github.com/angelo-angonezi).
+
 ## Installation
 
 `dog_label_fn` itself only needs patchworks' core deps (scipy). The

{patchworks-1.3.4 → patchworks-1.4.1}/docs/guide/ome_zarr_napari.md

@@ -39,9 +39,10 @@ existed.
 ## Convert any image to OME-ZARR
 
 `to_ome_zarr` accepts a dask/NumPy array, an existing `.zarr` store, an
-**Imaris `.ims`** file, or **any format** readable by
-[bioio](https://github.com/bioio-devs/bioio) (CZI, LIF, ND2, OME-TIFF, …). File
-inputs are read **lazily**.
+**Imaris `.ims`** file, **any format** readable by
+[bioio](https://github.com/bioio-devs/bioio) (CZI, LIF, ND2, OME-TIFF, …), or
+a **folder of single-plane TIFFs** (see below). File inputs are read
+**lazily**.
 
 ```python
 from patchworks.plugins.ome_zarr import to_ome_zarr
@@ -50,6 +51,51 @@ to_ome_zarr("scan.czi", "scan.zarr", n_levels=5)   # via bioio
 to_ome_zarr("scan.ims", "scan.zarr")               # Imaris, native HDF5
 ```
 
+### A folder of single-plane TIFFs
+
+Some acquisitions/stitching tools save **one TIFF per Z/C plane** instead of a
+single multi-page file, e.g.:
+
+```text
+sample_T0_Z000_C0_V0.tif
+sample_T0_Z000_C1_V0.tif
+sample_T0_Z001_C0_V0.tif
+sample_T0_Z001_C1_V0.tif
+...
+```
+
+Pass `sequence_pattern=` and let `source` be a glob over the folder instead of
+one file path. The pattern is a regex whose **named groups** map to axis
+labels — one file per group of indices:
+
+```python
+to_ome_zarr(
+    "sample/*.tif",
+    "sample.zarr",
+    sequence_pattern=r"_T(?P<T>\d+)_Z(?P<Z>\d+)_C(?P<C>\d+)_V\d+",
+    shard=True,  # recommended for large sequences, see Sharding below
+)
+```
+
+You don't need to order the named groups to match how the axes should come
+out: the result is always reordered to patchworks' `c, z, y, x` convention —
+required for `channel=` selection (`load_ome_zarr`/`tile_process` always read
+the channel from axis 0) — and a constant axis like `T0` above (one value, no
+real time series) is dropped automatically, the same way the bioio/Imaris
+readers already drop a singleton time axis.
+
+Each file becomes exactly **one dask chunk**, decoded lazily on access — no
+data is duplicated or eagerly loaded, so this scales to huge (multi-TB)
+sequences (built on `tifffile.TiffSequence`, the same mechanism Cellpose's
+own distributed pipeline uses). Pixel calibration is read automatically from
+the first file's own metadata — see [Pixel calibration](#pixel-calibration)
+below.
+
+Available on the cluster too: the Snakemake `convert` rule reads a
+`sequence_pattern:` key from the config (`input:` then being the glob), so a
+folder-of-TIFFs conversion runs through the same SLURM profile as any other
+input — see [Cluster usage](snakemake.md).
+
 !!! note "Imaris pyramids: rebuild (default) or reuse"
     `.ims` files carry their own resolution pyramid. By default `to_ome_zarr`
     reads only the **full-resolution** level and **builds a fresh NGFF pyramid**
@@ -64,9 +110,11 @@ to_ome_zarr("scan.ims", "scan.zarr")               # Imaris, native HDF5
 ### Pixel calibration
 
 The physical voxel size is read from the input — bioio's `physical_pixel_sizes`,
-the Imaris resolution metadata, or an existing OME-ZARR's scale — and written
-into the NGFF `coordinateTransformations` (in micrometers), so calibration is
-preserved regardless of input. Override or supply it for bare arrays with
+the Imaris resolution metadata, an existing OME-ZARR's scale, or (for a TIFF
+sequence) the first file's own ImageJ metadata (`spacing`/`unit`) or
+`XResolution`/`YResolution` tags — and written into the NGFF
+`coordinateTransformations` (in micrometers), so calibration is preserved
+regardless of input. Override or supply it for bare arrays with
 `pixel_size={"z": 2.0, "y": 0.32, "x": 0.32}`.
 
 ### Won't OOM

{patchworks-1.3.4 → patchworks-1.4.1}/src/patchworks/plugins/ome_zarr.py

@@ -9,14 +9,18 @@ Input handling, in order:
 * a ``.zarr`` path → read through :func:`patchworks.load_ome_zarr`;
 * a ``.ims`` (Imaris) path → read lazily with ``imaris-ims-file-reader``
   (HDF5, no JVM); install with ``pip install "patchworks[imaris]"``;
+* with ``sequence_pattern=`` set, a glob over a folder of single-plane TIFFs
+  → each file becomes one chunk of a lazy dask array via
+  :func:`tifffile.TiffSequence`, no data duplicated;
 * any other path (CZI, LIF, ND2, OME-TIFF, …) → opened lazily with
   `bioio <https://github.com/bioio-devs/bioio>`_.
 
 Pixel calibration (physical voxel size) is read from the input — bioio's
-``physical_pixel_sizes``, the Imaris resolution metadata, or an existing
-OME-ZARR's scale transform — and written into the NGFF ``coordinate
-Transformations`` so the µm/pixel sizing is preserved. Pass ``pixel_size=`` to
-override or to supply it for bare arrays.
+``physical_pixel_sizes``, the Imaris resolution metadata, an existing
+OME-ZARR's scale transform, or (for a TIFF sequence) the first file's own
+ImageJ/resolution tags — and written into the NGFF ``coordinateTransformations``
+so the µm/pixel sizing is preserved. Pass ``pixel_size=`` to override or to
+supply it for bare arrays.
 
 Downsampling uses strided (nearest-neighbour) subsampling — the correct,
 label-preserving choice — and only on **X and Y**; ``z`` (and channel/time)
@@ -620,6 +624,146 @@ def _open_bioio(path: str, scene: int) -> tuple[da.Array, str, PixelSize]:
     return arr, axes, pixel_size
 
 
+_UNIT_TO_UM = {
+    "micron": 1.0,
+    "um": 1.0,
+    "µm": 1.0,
+    "mm": 1000.0,
+    "cm": 10000.0,
+    "centimeter": 10000.0,
+    "inch": 25400.0,
+}
+# TIFF ResolutionUnit tag values (RESUNIT): 2 == INCH, 3 == CENTIMETER.
+_RESUNIT_TO_UM = {2: 25400.0, 3: 10000.0}
+
+
+def _tiff_pixel_size(path: str) -> PixelSize:
+    """Read physical voxel size from a TIFF file's own metadata.
+
+    Z comes from ImageJ metadata (``spacing`` + ``unit``), if present. X/Y
+    come from the page's ``XResolution``/``YResolution`` tags (pixels per
+    unit); the unit itself is taken from the ``ResolutionUnit`` tag when set
+    (plain TIFFs), or falls back to ImageJ metadata's ``unit`` — ImageJ
+    itself writes ``ResolutionUnit=NONE`` and keeps the unit as a string in
+    its metadata instead. Unrecognized units are ignored (treated as
+    uncalibrated).
+
+    Parameters
+    ----------
+    path : str
+        Path of one TIFF file in the sequence.
+
+    Returns
+    -------
+    dict
+        ``{axis: micrometers}`` for whichever axes could be determined
+        (empty if none).
+    """
+    import tifffile
+
+    pixel_size: PixelSize = {}
+    with tifffile.TiffFile(path) as tif:
+        ij = tif.imagej_metadata or {}
+        spacing = ij.get("spacing")
+        if spacing:
+            factor = _UNIT_TO_UM.get(str(ij.get("unit", "micron")).lower(), 1.0)
+            pixel_size["z"] = float(spacing) * factor
+
+        page = tif.pages[0]
+        unit_tag = page.tags.get("ResolutionUnit")
+        um_per_unit = _RESUNIT_TO_UM.get(
+            unit_tag.value if unit_tag else None
+        ) or _UNIT_TO_UM.get(str(ij.get("unit", "")).lower())
+        if um_per_unit is not None:
+            for axis, tag_name in (("y", "YResolution"), ("x", "XResolution")):
+                tag = page.tags.get(tag_name)
+                if tag and tag.value[0]:
+                    num, den = tag.value
+                    pixel_size[axis] = um_per_unit / (num / den)
+    return pixel_size
+
+
+def _open_tiff_sequence(
+    pattern: str, sequence_pattern: str
+) -> tuple[da.Array, str, PixelSize]:
+    """Open a folder of single-plane TIFFs as one lazy dask array.
+
+    Each file becomes exactly one chunk, decoded on access — no data is
+    duplicated or eagerly loaded. See
+    :func:`tifffile.TiffSequence`/``ZarrFileSequenceStore`` (the same
+    mechanism used by Cellpose's own distributed pipeline,
+    ``cellpose.contrib.distributed_segmentation.wrap_folder_of_tiffs``).
+
+    Parameters
+    ----------
+    pattern : str
+        Glob pattern matching all the files, e.g. ``"folder/*.tif"``.
+    sequence_pattern : str
+        Regular expression parsing each file name into axis labels and
+        indices, e.g. ``r"_T(?P<T>\\d+)_Z(?P<Z>\\d+)_C(?P<C>\\d+)_V\\d+"``.
+        Named groups become axis labels directly — the order they appear in
+        the *pattern* (not the filename) doesn't matter, the result is
+        always reordered to patchworks' ``tczyx`` convention and any
+        singleton non-spatial axis (e.g. a constant ``T0``) is dropped, same
+        as the bioio/Imaris readers, so a real channel axis always ends up
+        first (required by :func:`patchworks.load_ome_zarr`).
+
+    Returns
+    -------
+    tuple
+        ``(array, axes, pixel_size)`` — a lazy dask array, its axes string
+        (parsed axes, canonicalised, + trailing ``"yx"``) and a
+        ``{axis: micrometers}`` calibration dict read from the first file.
+    """
+    try:
+        import tifffile
+    except ImportError as exc:
+        raise ImportError(
+            "reading a folder of TIFFs requires tifffile. Install it with:\n"
+            "    pip install tifffile\n"
+            "(already pulled in by patchworks[bioio] via bioio-tifffile)."
+        ) from exc
+
+    ts = tifffile.TiffSequence(pattern, pattern=sequence_pattern)
+    arr = da.from_zarr(zarr.open(store=ts.aszarr()))
+    full_axes = ts.axes.lower() + "yx"
+
+    # Canonicalise to patchworks' tczyx axis order regardless of the order
+    # sequence_pattern's groups happen to appear in (unrecognised axes sort
+    # last, before y/x).
+    order = sorted(
+        range(len(full_axes)),
+        key=lambda i: (
+            _DEFAULT_ORDER.index(full_axes[i])
+            if full_axes[i] in _DEFAULT_ORDER
+            else len(_DEFAULT_ORDER)
+        ),
+    )
+    arr = arr.transpose(order)
+    ordered_axes = "".join(full_axes[i] for i in order)
+
+    # Drop singleton non-spatial axes (e.g. a constant T), matching
+    # _open_bioio/_open_imaris, so a real channel axis lands at position 0.
+    keep = [
+        i
+        for i, a in enumerate(ordered_axes)
+        if a in _SPATIAL_AXES or arr.shape[i] > 1
+    ]
+    index = tuple(slice(None) if i in keep else 0 for i in range(arr.ndim))
+    arr = arr[index]
+    axes = "".join(ordered_axes[i] for i in keep)
+
+    pixel_size = _tiff_pixel_size(ts[0])
+    logger.info(
+        "tifffile sequence opened %s files as %s %s cal=%s",
+        len(ts),
+        axes,
+        arr.shape,
+        pixel_size,
+    )
+    return arr, axes, pixel_size
+
+
 def _open_imaris(path: str, level: int = 0) -> tuple[da.Array, str, PixelSize]:
     """Open one Imaris ``.ims`` resolution level lazily.
 
@@ -769,12 +913,14 @@ def _to_dask(
     source: Union[da.Array, np.ndarray, str, Path],
     axes: Union[str, None],
     scene: int,
+    sequence_pattern: Union[str, None] = None,
 ) -> tuple[da.Array, str, PixelSize]:
     """Resolve *source* into a lazy ``(array, axes, pixel_size)`` triple.
 
     Dispatches by type: dask/NumPy arrays pass through; ``.zarr`` paths use the
-    OME-ZARR loader; ``.ims`` paths use the Imaris reader; anything else uses
-    bioio.
+    OME-ZARR loader; ``.ims`` paths use the Imaris reader; a *sequence_pattern*
+    treats *source* as a glob over a folder of single-plane TIFFs; anything
+    else uses bioio.
 
     Parameters
     ----------
@@ -784,6 +930,10 @@ def _to_dask(
         Explicit axes, or ``None`` to infer them.
     scene : int
         Scene index for bioio inputs.
+    sequence_pattern : str or None
+        When given, *source* is a glob pattern over a folder of single-plane
+        TIFFs and this is the filename-parsing regex; see
+        :func:`_open_tiff_sequence`.
 
     Returns
     -------
@@ -796,6 +946,9 @@ def _to_dask(
         return da.asarray(source), axes or _default_axes(source.ndim), {}
 
     path = str(source)
+    if sequence_pattern is not None:
+        arr, detected, ps = _open_tiff_sequence(path, sequence_pattern)
+        return arr, axes or detected, ps
     if path.endswith(".zarr"):
         arr = load_ome_zarr(source, channel=None)
         ax = axes or _default_axes(arr.ndim)
@@ -815,6 +968,7 @@ def to_ome_zarr(
     axes: Union[str, None] = None,
     pixel_size: Union[PixelSize, tuple, None] = None,
     scene: int = 0,
+    sequence_pattern: Union[str, None] = None,
     n_levels: int = 5,
     downscale: int = 2,
     chunks: Union[tuple[int, ...], None] = None,
@@ -826,7 +980,8 @@ def to_ome_zarr(
     """Write *source* as a pyramidal, calibrated OME-ZARR store.
 
     *source* may be a dask/NumPy array, a ``.zarr`` store, an Imaris ``.ims``
-    file, or any image format readable by bioio (CZI, LIF, ND2, OME-TIFF, …).
+    file, any image format readable by bioio (CZI, LIF, ND2, OME-TIFF, …), or
+    (with *sequence_pattern* set) a glob over a folder of single-plane TIFFs.
     File inputs are read lazily; the pyramid is built level-by-level from disk
     with bounded chunks, so the full volume never needs to fit in RAM. Only
     ``x``/``y`` are downsampled; ``z`` (and channel/time) stay full-resolution.
@@ -847,6 +1002,18 @@ def to_ome_zarr(
         arrays.
     scene : int, optional
         Scene index for multi-scene bioio files.
+    sequence_pattern : str, optional
+        When given, *source* is treated as a glob pattern over a folder of
+        single-plane TIFFs (e.g. ``"folder/*.tif"``) instead of a single
+        file, and this is the regex parsing each file name into axis labels
+        and indices via named groups, e.g.
+        ``r"_T(?P<T>\\d+)_Z(?P<Z>\\d+)_C(?P<C>\\d+)_V\\d+"``. Each file
+        becomes exactly one chunk, read lazily on access (no data
+        duplicated) — see :func:`tifffile.TiffSequence`. The result is
+        always reordered to patchworks' ``czyx`` convention regardless of
+        the order named groups appear in the pattern, and any singleton
+        non-spatial axis (e.g. a constant ``T0``) is dropped, so a real
+        channel axis always ends up first.
     n_levels : int, optional
         Maximum number of pyramid levels including full resolution.
     downscale : int, optional
@@ -881,6 +1048,13 @@ def to_ome_zarr(
     >>> from patchworks.plugins.ome_zarr import to_ome_zarr
     >>> to_ome_zarr("scan.ims", "scan.zarr", n_levels=4)
     'scan.zarr'
+    >>> to_ome_zarr(
+    ...     "ZT18_Male4_Left/*.tif",
+    ...     "ZT18_Male4_Left.zarr",
+    ...     sequence_pattern=r"_T(?P<T>\\d+)_Z(?P<Z>\\d+)_C(?P<C>\\d+)_V\\d+",
+    ...     shard=True,
+    ... )  # doctest: +SKIP
+    'ZT18_Male4_Left.zarr'
     """
     if downscale < 2:
         raise ValueError("downscale must be >= 2")
@@ -908,7 +1082,7 @@ def to_ome_zarr(
                 exc,
             )
 
-    arr, axes, detected = _to_dask(source, axes, scene)
+    arr, axes, detected = _to_dask(source, axes, scene, sequence_pattern)
     if len(axes) != arr.ndim:
         raise ValueError(
             f"axes {axes!r} has {len(axes)} entries but array is {arr.ndim}-D"

{patchworks-1.3.4 → patchworks-1.4.1}/tests/test_ome_zarr.py

@@ -79,6 +79,72 @@ def test_non_spatial_axis_not_downsampled(tmp_path):
     assert load_ome_zarr(out, channel=None, level=1).shape == (3, 8, 8)
 
 
+def test_tiff_sequence_conversion(tmp_path):
+    """A folder of single-plane TIFFs is wrapped lazily and converted.
+
+    The filename pattern lists Z before C, but the output must still come
+    out channel-first (patchworks' tczyx convention), since load_ome_zarr /
+    tile_process hard-assume axis 0 is the channel axis.
+    """
+    tifffile = pytest.importorskip("tifffile")
+    n_z, n_c, size = 3, 2, 8
+    for z in range(n_z):
+        for c in range(n_c):
+            img = np.full((size, size), z * 10 + c, dtype="uint16")
+            tifffile.imwrite(
+                tmp_path / f"sample_Z{z:03d}_C{c}_V0.tif",
+                img,
+                resolution=(20000.0, 20000.0),
+                resolutionunit="CENTIMETER",
+            )
+
+    out = tmp_path / "out.zarr"
+    to_ome_zarr(
+        str(tmp_path / "*.tif"),
+        out,
+        sequence_pattern=r"_Z(?P<Z>\d+)_C(?P<C>\d+)_V\d+",
+        n_levels=1,
+    )
+
+    result = np.asarray(load_ome_zarr(out, channel=None))
+    assert result.shape == (n_c, n_z, size, size)  # channel-first, not z-first
+    # each plane's constant value encodes its (z, c) position.
+    assert (
+        result[:, :, 0, 0]
+        == [[z * 10 + c for z in range(n_z)] for c in range(n_c)]
+    ).all()
+    assert _level_scale(out, 0) == pytest.approx([1.0, 1.0, 0.5, 0.5])
+
+    # per-channel selection picks the right plane regardless of pattern order.
+    ch1 = np.asarray(load_ome_zarr(out, channel=1))
+    assert ch1.shape == (n_z, size, size)
+    assert (ch1[:, 0, 0] == [z * 10 + 1 for z in range(n_z)]).all()
+
+
+def test_tiff_sequence_drops_singleton_time_axis(tmp_path):
+    """A constant T in the pattern is dropped, keeping channel at axis 0."""
+    tifffile = pytest.importorskip("tifffile")
+    n_z, n_c, size = 2, 3, 8
+    for z in range(n_z):
+        for c in range(n_c):
+            img = np.full((size, size), z * 10 + c, dtype="uint16")
+            tifffile.imwrite(tmp_path / f"sample_T0_Z{z:03d}_C{c}_V0.tif", img)
+
+    out = tmp_path / "out.zarr"
+    to_ome_zarr(
+        str(tmp_path / "*.tif"),
+        out,
+        sequence_pattern=r"_T(?P<T>\d+)_Z(?P<Z>\d+)_C(?P<C>\d+)_V\d+",
+        n_levels=1,
+    )
+
+    result = np.asarray(load_ome_zarr(out, channel=None))
+    assert result.shape == (n_c, n_z, size, size)  # no leftover T axis
+    ch2 = np.asarray(load_ome_zarr(out, channel=2))
+    assert ch2.shape == (n_z, size, size)
+    assert (ch2[:, 0, 0] == [z * 10 + 2 for z in range(n_z)]).all()
+
+
 def test_axes_length_mismatch(tmp_path):
     with pytest.raises(ValueError):
         to_ome_zarr(

{patchworks-1.3.4 → patchworks-1.4.1}/workflow/config/config.yaml

@@ -5,7 +5,9 @@
 # GPUs run in parallel; the merge is a single CPU job.
 
 # ---- input / output ---------------------------------------------------------
-input: "/path/to/scan.ims" # .ims, .czi, .lif, .nd2, ome-tiff, or an .zarr
+input: "/path/to/scan.ims" # .ims, .czi, .lif, .nd2, ome-tiff, .zarr, or a
+#                             glob (e.g. "/path/*.tif") when sequence_pattern
+#                             is set below
 work_dir: "/path/to/results" # everything is written under here
 # Final outputs (under work_dir):
 #   image.zarr            — converted, pyramidal OME-ZARR
@@ -15,6 +17,10 @@ work_dir: "/path/to/results" # everything is written under here
 reuse_pyramid: false # for .ims: copy its own pyramid (fast); else rebuild
 convert_chunks: null # null → patchworks' bounded auto chunks; or [c,z,y,x]
 shard: false # true → pack chunks into shards (zarr v3; fewer files)
+# For a folder of single-plane TIFFs (input: a glob), set the filename regex
+# below; named groups become axes, e.g. one file per Z/C plane:
+# sequence_pattern: '_T(?P<T>\d+)_Z(?P<Z>\d+)_C(?P<C>\d+)_V\d+'
+sequence_pattern: null
 # Re-running reuses an existing image.zarr automatically (skips conversion).
 # To force a rebuild: delete image.zarr, or `snakemake --forcerun convert`.
 

{patchworks-1.3.4 → patchworks-1.4.1}/workflow/scripts/_pw.py

@@ -152,7 +152,9 @@ def build_fn(cfg):
     if dilate:
         from patchworks import dilate_labels
 
-        fn = dilate_labels(fn, iterations=dilate, use_gpu=cfg.get("dilate_gpu", False))
+        fn = dilate_labels(
+            fn, iterations=dilate, use_gpu=cfg.get("dilate_gpu", False)
+        )
 
     return fn
 

{patchworks-1.3.4 → patchworks-1.4.1}/workflow/scripts/convert.py

@@ -16,6 +16,7 @@ chunks = cfg.get("convert_chunks")
 to_ome_zarr(
     cfg["input"],
     str(snakemake.output[0]).removesuffix("/zarr.json"),  # noqa: F821
+    sequence_pattern=cfg.get("sequence_pattern"),
     chunks=tuple(chunks) if chunks else None,
     shard=bool(cfg.get("shard", False)),
     reuse_pyramid=bool(cfg.get("reuse_pyramid", False)),