micro-reader 0.0.1__tar.gz → 0.0.3__tar.gz

{micro_reader-0.0.1 → micro_reader-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: micro-reader
-Version: 0.0.1
+Version: 0.0.3
 Summary: Low-level, memory-bounded pixel reading for microscopy formats: read exactly the rectangle you ask for.
 Project-URL: Homepage, https://github.com/bugraoezdemir/micro-reader
 Project-URL: Source, https://github.com/bugraoezdemir/micro-reader
@@ -192,6 +192,10 @@ This returns the images matching the requested indices. For example, if a file c
 `micro-reader` uses these index names when a format provides the corresponding dimensions:
 
 ```text
+plate
+acquisition
+well
+field
 scene
 tile
 rotation
@@ -203,6 +207,7 @@ emission
 excitation
 lifetime
 sequence
+part
 resolution
 ```
 
@@ -425,6 +430,37 @@ The tiles are composed according to the format's stitching information.
 
 Currently, stitching is supported for **CZI only**.
 
+## Plates
+
+High-content screening plates are opened through their index file, for example `Index.idx.xml` for Harmony. Each field is one image, indexed by `plate`, `acquisition`, `well` and `field`.
+
+The plate layout is available through:
+
+```python
+plate = f.plates[0]
+plate.wells[0].name
+# "B2"
+```
+
+Images can be reached through the wells, or selected by well:
+
+```python
+image = plate.wells[0].fields[0].image
+f.select(well=plate.well_index("B2"))
+```
+
+`f.outline()` shows the plates, wells and fields of a file.
+
+Zeiss CZI plates are read as plates with `plates=True`, and as scenes otherwise:
+
+```python
+f = micro_reader.open("plate.czi", plates=True)
+```
+
+Missing plane files are read as zeros, with a `MissingImagesWarning`. A field without any of its files is left out.
+
+When the channels of a field have different z sampling, `image.parts` gives one image per group of channels that share it.
+
 ## Metadata
 
 Pixel and channel metadata are available through:
@@ -489,12 +525,14 @@ The position is omitted when it cannot be determined reliably.
 | Format               | Notes                                                 |
 | -------------------- | ----------------------------------------------------- |
 | **TIFF / BigTIFF**   | TIFF, OME-TIFF, ImageJ, LSM, STK and TIFF pyramids    |
+| **OME companion**    | `.companion.ome` sets of OME-TIFF files               |
 | **CZI**              | Zeiss CZI, including mosaic tiles and stitched scenes |
 | **ND2**              | Nikon ND2, including legacy JPEG 2000 ND2             |
 | **Leica**            | LIF, LOF, LIFEXT, XLIF, XLEF, XLCF                    |
 | **OIR**              | Olympus / Evident FluoView                            |
 | **Imaris**           | `.ims`, with the `[ims]` extra                        |
 | **MRC**              | Common MRC modes and extended headers                 |
+| **Plates (HCS)**     | Harmony, MetaXpress, IN Cell and CellVoyager plates   |
 | **JPEG / PNG / BMP** | Common raster image formats                           |
 
 All formats use the same `File`, `Image`, axes, indexing, metadata, and reading interface wherever possible.
@@ -534,6 +572,16 @@ Additional codecs such as JPEG XR and LERC are available with:
 pip install "micro-reader[codecs]"
 ```
 
+### OME companion files
+
+An OME companion file (`.companion.ome`) describes images stored in separate TIFF files. Opening the companion reads all those images and groups them together as one `t c z y x` image:
+
+```python
+f = micro_reader.open("data.companion.ome")
+```
+
+Opening one of the TIFF files on its own reads only that file, with its pixel sizes, channels and position taken from the companion. If the companion is missing, the file is read without pixel sizes, with a `CompanionWarning`.
+
 ### CZI
 
 CZI files support:
@@ -677,7 +725,23 @@ You can also set it through the environment:
 export MICRO_READER_MEMORY_LIMIT=8GB
 ```
 
-For uncompressed data, only the requested rows and columns are read.
+The limit covers the blocks being decoded at once. Decoded blocks are also kept for reuse,
+so a block shared by neighbouring reads is not decoded twice. All open files share one
+cache limit, 256 MB by default:
+
+```python
+micro_reader.set_cache_limit("512MB")
+micro_reader.cache_limit()
+micro_reader.cached_bytes()     # what the caches hold now
+```
+
+or `export MICRO_READER_CACHE_LIMIT=512MB`. The process then uses at most the memory
+limit plus the cache limit for decoding, however many files are open. One exception: a
+single picture (PNG, JPEG, ... up to 256 MB) is kept decoded by its own open file.
+
+For uncompressed data, only the requested rows are read. When you ask for part of
+each row, the rest of the row is read along with it if that is faster than reading the
+rows one by one.
 
 Compressed data is stored in blocks such as TIFF strips or tiles, CZI subblocks, and ND2 frames. Small blocks are decoded as a whole. Blocks larger than 16 MB are read in parts where the codec allows it.
 

{micro_reader-0.0.1 → micro_reader-0.0.3}/README.md

@@ -140,6 +140,10 @@ This returns the images matching the requested indices. For example, if a file c
 `micro-reader` uses these index names when a format provides the corresponding dimensions:
 
 ```text
+plate
+acquisition
+well
+field
 scene
 tile
 rotation
@@ -151,6 +155,7 @@ emission
 excitation
 lifetime
 sequence
+part
 resolution
 ```
 
@@ -373,6 +378,37 @@ The tiles are composed according to the format's stitching information.
 
 Currently, stitching is supported for **CZI only**.
 
+## Plates
+
+High-content screening plates are opened through their index file, for example `Index.idx.xml` for Harmony. Each field is one image, indexed by `plate`, `acquisition`, `well` and `field`.
+
+The plate layout is available through:
+
+```python
+plate = f.plates[0]
+plate.wells[0].name
+# "B2"
+```
+
+Images can be reached through the wells, or selected by well:
+
+```python
+image = plate.wells[0].fields[0].image
+f.select(well=plate.well_index("B2"))
+```
+
+`f.outline()` shows the plates, wells and fields of a file.
+
+Zeiss CZI plates are read as plates with `plates=True`, and as scenes otherwise:
+
+```python
+f = micro_reader.open("plate.czi", plates=True)
+```
+
+Missing plane files are read as zeros, with a `MissingImagesWarning`. A field without any of its files is left out.
+
+When the channels of a field have different z sampling, `image.parts` gives one image per group of channels that share it.
+
 ## Metadata
 
 Pixel and channel metadata are available through:
@@ -437,12 +473,14 @@ The position is omitted when it cannot be determined reliably.
 | Format               | Notes                                                 |
 | -------------------- | ----------------------------------------------------- |
 | **TIFF / BigTIFF**   | TIFF, OME-TIFF, ImageJ, LSM, STK and TIFF pyramids    |
+| **OME companion**    | `.companion.ome` sets of OME-TIFF files               |
 | **CZI**              | Zeiss CZI, including mosaic tiles and stitched scenes |
 | **ND2**              | Nikon ND2, including legacy JPEG 2000 ND2             |
 | **Leica**            | LIF, LOF, LIFEXT, XLIF, XLEF, XLCF                    |
 | **OIR**              | Olympus / Evident FluoView                            |
 | **Imaris**           | `.ims`, with the `[ims]` extra                        |
 | **MRC**              | Common MRC modes and extended headers                 |
+| **Plates (HCS)**     | Harmony, MetaXpress, IN Cell and CellVoyager plates   |
 | **JPEG / PNG / BMP** | Common raster image formats                           |
 
 All formats use the same `File`, `Image`, axes, indexing, metadata, and reading interface wherever possible.
@@ -482,6 +520,16 @@ Additional codecs such as JPEG XR and LERC are available with:
 pip install "micro-reader[codecs]"
 ```
 
+### OME companion files
+
+An OME companion file (`.companion.ome`) describes images stored in separate TIFF files. Opening the companion reads all those images and groups them together as one `t c z y x` image:
+
+```python
+f = micro_reader.open("data.companion.ome")
+```
+
+Opening one of the TIFF files on its own reads only that file, with its pixel sizes, channels and position taken from the companion. If the companion is missing, the file is read without pixel sizes, with a `CompanionWarning`.
+
 ### CZI
 
 CZI files support:
@@ -625,7 +673,23 @@ You can also set it through the environment:
 export MICRO_READER_MEMORY_LIMIT=8GB
 ```
 
-For uncompressed data, only the requested rows and columns are read.
+The limit covers the blocks being decoded at once. Decoded blocks are also kept for reuse,
+so a block shared by neighbouring reads is not decoded twice. All open files share one
+cache limit, 256 MB by default:
+
+```python
+micro_reader.set_cache_limit("512MB")
+micro_reader.cache_limit()
+micro_reader.cached_bytes()     # what the caches hold now
+```
+
+or `export MICRO_READER_CACHE_LIMIT=512MB`. The process then uses at most the memory
+limit plus the cache limit for decoding, however many files are open. One exception: a
+single picture (PNG, JPEG, ... up to 256 MB) is kept decoded by its own open file.
+
+For uncompressed data, only the requested rows are read. When you ask for part of
+each row, the rest of the row is read along with it if that is faster than reading the
+rows one by one.
 
 Compressed data is stored in blocks such as TIFF strips or tiles, CZI subblocks, and ND2 frames. Small blocks are decoded as a whole. Blocks larger than 16 MB are read in parts where the codec allows it.
 

{micro_reader-0.0.1 → micro_reader-0.0.3}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "micro-reader"
-version = "0.0.1"
+version = "0.0.3"
 description = "Low-level, memory-bounded pixel reading for microscopy formats: read exactly the rectangle you ask for."
 readme = "README.md"
 requires-python = ">=3.11"

{micro_reader-0.0.1 → micro_reader-0.0.3}/src/micro_reader/__init__.py

@@ -4,20 +4,25 @@
 >>> with micro_reader.open("image.mrc") as f:
 ...     block = f.images[0].read(z=0, y=slice(0, 512))
 """
-from ._base import INDEX_NAMES, SAMPLED_INDICES, File, Image, ReadStats
+from ._base import INDEX_NAMES, PLATE_INDICES, SAMPLED_INDICES, File, Image, ReadStats
 from ._metadata import Channel, Metadata, Quantity
-from ._errors import CorruptFileError, MicroReaderError, UnsupportedFormatError
+from ._plate import Acquisition, Field, Plate, Well
+from ._errors import (CompanionWarning, CorruptFileError, MicroReaderError,
+                      MissingImagesWarning, UnsupportedFormatError)
 from ._memory import MemoryLimitError, memory_limit, set_memory_limit
+from ._io import cache_limit, cached_bytes, set_cache_limit
 from ._pool import decode_threads, set_decode_threads
 from ._registry import BACKENDS, open  # noqa: A004 - the public entry point
 from .backends.czi import DamagedFileWarning
 
-__version__ = "0.0.1"
+__version__ = "0.0.3"
 
 __all__ = [
-    "open", "File", "Image", "INDEX_NAMES", "SAMPLED_INDICES", "ReadStats", "BACKENDS",
-    "Metadata", "Channel", "Quantity",
+    "open", "File", "Image", "INDEX_NAMES", "PLATE_INDICES", "SAMPLED_INDICES", "ReadStats",
+    "BACKENDS", "Metadata", "Channel", "Quantity", "Plate", "Well", "Field", "Acquisition",
     "decode_threads", "set_decode_threads", "memory_limit", "set_memory_limit",
+    "cache_limit", "set_cache_limit", "cached_bytes",
     "MicroReaderError", "UnsupportedFormatError", "CorruptFileError", "MemoryLimitError", "DamagedFileWarning",
+    "MissingImagesWarning", "CompanionWarning",
     "__version__",
 ]

{micro_reader-0.0.1 → micro_reader-0.0.3}/src/micro_reader/_base.py

@@ -65,9 +65,12 @@ class Image:
         self.name = name if name is not None else f"image{position}"
         #: which image of the file: {"scene": .., "tile": .., "rotation": ..,
         #: "view": .., "resolution": ..} -- only the indices the file has
-        #: ("resolution" only when opened with resolutions="all")
+        #: (plates: "well", "field", ... instead of "scene"; see INDEX_NAMES)
         self.index = dict(index) if index is not None else {"scene": self._position}
         self._resolutions: Optional[list] = None
+        #: a ragged image's rectangular parts / a part's whole (see ``parts``)
+        self._parts: Optional[list] = None
+        self._whole: Optional["Image"] = None
         self._metadata = None           # backends set a _metadata.Metadata
         #: None: any rectangle is read exactly.  Otherwise the smallest block
         #: the backend reads at once (e.g. a compressed tile or strip).
@@ -108,6 +111,55 @@ class Image:
         (``resolutions`` holds more than the image itself)."""
         return len(self.resolutions) > 1
 
+    # An image is either a pyramid (the logical image: level 0, also when the
+    # file stores no lower levels) or a lower resolution level of one.
+
+    @property
+    def is_pyramid(self) -> bool:
+        """Whether this image is a pyramid, the image itself at full
+        resolution, rather than a lower resolution level of one.  True for
+        every image without lower levels too."""
+        return self.pyramid is self
+
+    @property
+    def level(self) -> int:
+        """This image's resolution level in its pyramid: 0 for the pyramid
+        itself (full resolution), 1, 2, ... for lower levels."""
+        for n, member in enumerate(self.resolutions):
+            if member is self:
+                return n
+        raise RuntimeError(f"{self.name!r} is not in its own pyramid")
+
+    @property
+    def pyramid(self) -> "Image":
+        """The pyramid this image belongs to: itself at level 0, the
+        full-resolution image for a lower level."""
+        return self.resolutions[0]
+
+    # A ragged image: its channels do not share one sampling (e.g. one channel
+    # a z-stack, the others single planes; docs/AXES.md).  It is a whole made of
+    # rectangular parts, each an ordinary lazy Image with its own z / t and
+    # metadata.  The whole reads as one array, padded with zeros, when the
+    # parts lie on a common grid; otherwise reading it raises.
+
+    @property
+    def parts(self) -> list:
+        """This image's rectangular parts, channel groups that share one
+        sampling: several for a ragged image, just this image otherwise."""
+        return list(self._parts) if self._parts else [self]
+
+    @property
+    def is_ragged(self) -> bool:
+        """Whether this image's channels do not share one sampling
+        (``parts`` holds more than one part)."""
+        return len(self.parts) > 1
+
+    @property
+    def whole(self) -> "Image":
+        """The image this part belongs to: itself for a whole (or an image
+        that is not ragged)."""
+        return self._whole if self._whole is not None else self
+
     @property
     def resolutions(self) -> list:
         """This image's pyramid, full resolution first: one image per
@@ -177,12 +229,18 @@ class Image:
 
 #: the indices that say which image of a file an image is, outermost first.
 #: Arrays have only t c z y x (+ s); every other dimension is an index:
-#: illumination (light-sheet sides), phase, block (CZI), emission / excitation
-#: (spectral scans; the wavelength in the image's metadata), lifetime (FLIM
-#: bins) and sequence (a dimension the file does not name; open(...,
+#: plate, acquisition (imaging run), well and field for plates (docs/HCS.md;
+#: on a plate, well and field take the place of scene), illumination
+#: (light-sheet sides), phase, block (CZI), emission / excitation (spectral
+#: scans; the wavelength in the image's metadata), lifetime (FLIM bins) and
+#: sequence (a dimension the file does not name; open(...,
 #: rename={"sequence": "z"}) makes it an axis)
-INDEX_NAMES = ("scene", "tile", "rotation", "view", "illumination", "phase", "block",
-               "emission", "excitation", "lifetime", "sequence", "resolution")
+INDEX_NAMES = ("plate", "acquisition", "well", "field",
+               "scene", "tile", "rotation", "view", "illumination", "phase", "block",
+               "emission", "excitation", "lifetime", "sequence", "part", "resolution")
+#: the indices of plates (docs/HCS.md), on every image of a plate;
+#: ``plate`` and ``acquisition`` are 0 when a file holds one of each
+PLATE_INDICES = ("plate", "acquisition", "well", "field")
 #: the indices that are not frames of the image but dimensions sampled at the
 #: same points (what other readers keep as array axes)
 SAMPLED_INDICES = ("illumination", "phase", "block", "emission", "excitation", "lifetime",
@@ -198,13 +256,24 @@ class File:
 
     backend: str = "base"
 
-    def __init__(self, path: str, images: Sequence[Image]):
+    def __init__(self, path: str, images: Sequence[Image], plates: Sequence = ()):
         self.path = str(path)
         self.images = list(images)
+        #: the plates the images belong to (``_plate.Plate``), like ``images``
+        #: a list; empty for a file that is not a plate (docs/HCS.md)
+        self.plates = list(plates)
         if not getattr(self, "format", None):
             #: the format variant; the backend's name unless it reads several
             self.format = self.backend
         self._index_resolutions()
+        self._bind_plates()
+
+    def _bind_plates(self) -> None:
+        """Fill in each plate's wells -> fields -> images from ``images``
+        (again whenever ``images`` is replaced, e.g. by ``open(axes=...)``)."""
+        if self.plates:
+            from ._plate import navigate
+            self.plates = navigate(self.plates, self.images)
 
     def _index_resolutions(self) -> None:
         """``resolution`` is in every index when the file stores a pyramid of
@@ -235,9 +304,10 @@ class File:
         values it takes, e.g. ``{"scene": 2, "tile": 5}``.  Not a total and
         not a shape: with 3 tiles in scene 0 and 5 in scene 1, tile indices
         run 0-2 and 0-4, so the range is 5 (not 8, and scene 0 has only 3).
-        Tile, rotation and view indices count from 0, so the values are
-        ``range(n)``; scene indices are the file's own (a CZI's may skip
-        numbers), so for scenes it is the number of distinct values."""
+        Tile, rotation, view, well and field indices count from 0, so the
+        values are ``range(n)``; scene indices are the file's own (a CZI's
+        may skip numbers), so for scenes it is the number of distinct
+        values.  A well index is a place in ``plates[p].wells``."""
         counts = {}
         for name in INDEX_NAMES:
             values = {image.index[name] for image in self.images if name in image.index}
@@ -263,6 +333,28 @@ class File:
     def __exit__(self, *exc) -> None:
         self.close()
 
-    def __repr__(self) -> str:
+    def outline(self) -> str:
+        """The file's structure (docs/AXES.md): its plates, wells and fields,
+        or its scenes, each with its tiles, the orthogonal indices that vary
+        in it (``x view 2``), the image shape and the pyramid levels."""
+        from ._outline import outline_lines
+        return "\n".join([self._header()] + ["  " + line for line in outline_lines(self)])
+
+    def _header(self) -> str:
+        plates = f" plates={len(self.plates)}" if self.plates else ""
         return (f"<micro_reader.File {self.path!r} backend={self.backend!r} "
-                f"format={self.format!r} images={len(self.images)}>")
+                f"format={self.format!r} images={len(self.images)}{plates}>")
+
+    #: outline lines shown by ``repr``; ``outline()`` shows all
+    REPR_LINES = 6
+
+    def __repr__(self) -> str:
+        from ._outline import outline_lines
+        try:
+            lines = outline_lines(self)
+        except Exception:                               # a repr never fails
+            return self._header()
+        shown = lines[:self.REPR_LINES]
+        if len(lines) > len(shown):
+            shown.append(f"... {len(lines) - len(shown)} more lines: .outline()")
+        return "\n".join([self._header()] + ["  " + line for line in shown])

{micro_reader-0.0.1 → micro_reader-0.0.3}/src/micro_reader/_errors.py

@@ -12,3 +12,14 @@ class UnsupportedFormatError(MicroReaderError):
 
 class CorruptFileError(MicroReaderError):
     """The file is recognised but inconsistent, e.g. truncated."""
+
+
+class MissingImagesWarning(UserWarning):
+    """A file set lists images whose files are missing; they are left out
+    (docs/HCS.md)."""
+
+
+class CompanionWarning(UserWarning):
+    """An OME companion set's metadata came from somewhere other than the file
+    opened, or a member path had to be resolved differently than written
+    (docs/COMPANION.md)."""