kuva-reader 1.0.4__tar.gz → 1.1.6__tar.gz

kuva_reader-1.1.6/.gitignore

@@ -0,0 +1,134 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+*.ipynb
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Torch stuff
+lightning_logs/
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# VSCode
+.vscode
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# Direnv stuff
+.direnv/
+.envrc
+
+# Poetry
+# poetry.lock
+
+# Supervisord
+supervisord.log
+supervisord.pid
+
+# Debug folders
+_debug/
+
+# Kuva data
+*.tif
+*.tiff
+*.npy
+hyperfield*.json
+
+# Do not ignore Kuva files in the test_data directory
+!kuva-reader/tests/test_data/**

{kuva_reader-1.0.4 → kuva_reader-1.1.6}/PKG-INFO

@@ -1,24 +1,16 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: kuva-reader
-Version: 1.0.4
+Version: 1.1.6
 Summary: Manipulate the Kuva Space image and metadata formats
-License: MIT
-Author: Guillem Ballesteros
-Author-email: guillem@kuvaspace.com
-Requires-Python: >=3.10,<=3.13
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Requires-Dist: kuva-geometry
-Requires-Dist: kuva-metadata
-Requires-Dist: numpy (>=1.26.4,<2.0.0)
-Requires-Dist: numpy-quaternion (>=2022.4.4,<2023.0.0)
-Requires-Dist: pint (>=0.22,<0.23)
-Requires-Dist: rasterio (>=1.4.1,<2.0.0)
-Requires-Dist: rioxarray (>=0.12.4,<0.13.0)
-Requires-Dist: xarray (>=2022.12.0,<2023.0.0)
+Author-email: Guillem Ballesteros <guillem@kuvaspace.com>, Lennert Antson <lennert.antson@kuvaspace.com>, Arthur Vandenhoeke <arthur.vandenhoeke@kuvaspace.com>, Olli Eloranta <olli.eloranta@kuvaspace.com>
+License-Expression: MIT
+Requires-Python: <=3.13,>=3.10
+Requires-Dist: kuva-geometry<2.0.0,>=1.0.1
+Requires-Dist: kuva-metadata<2.0.0,>=1.1.1
+Requires-Dist: numpy-quaternion>=2023.4.4
+Requires-Dist: numpy>=1.26.4
+Requires-Dist: pint<1.0.0,>=0.22
+Requires-Dist: rasterio<2,>=1.4.3
 Description-Content-Type: text/markdown
 
 <div align="center">
@@ -36,7 +28,7 @@ The Kuva Space images are in GeoTIFF format. The products consist of an image or
 images along with its metadata to give all the necessary information to use the products. 
 The metadata lives either in a Kuva Space database, or alternatively in a sidecar JSON file. 
 
-This library allows the reading of the image GeoTIFFs into `xarray.Dataset` objects that 
+This library allows the reading of the image GeoTIFFs into `rasterio.DatasetReader` objects that 
 allow convenient raster manipulations, along with their `kuva-metadata` metadata objects.
 
 # Installation
@@ -56,7 +48,7 @@ pip install kuva-reader
 This is a minimal example that allows you to read and print the image shape of a L2 product.
 
 The result product is in this case an L2A product (as seen from the folder name).
-The loaded product is stored in a `rioxarray` object, which contains extensive GIS functionalities [(examples for usage)](https://corteva.github.io/rioxarray/stable/examples/examples.html).
+The loaded product is stored in a `rasterio.DatasetReader` object, which contains extensive GIS functionalities [(examples for usage)](https://rasterio.readthedocs.io/en/stable/api/rasterio.io.html#rasterio.io.DatasetReader).
 
 ```python
 from kuva_reader import read_product
@@ -130,4 +122,3 @@ The `kuva-reader` project software is under the [MIT license](https://github.com
 # Status of unit tests
 
 [![Unit tests for kuva-reader](https://github.com/KuvaSpace/kuva-data-processing/actions/workflows/test-kuva-reader.yml/badge.svg)](https://github.com/KuvaSpace/kuva-data-processing/actions/workflows/test-kuva-reader.yml)
-

{kuva_reader-1.0.4 → kuva_reader-1.1.6}/README.md

@@ -13,7 +13,7 @@ The Kuva Space images are in GeoTIFF format. The products consist of an image or
 images along with its metadata to give all the necessary information to use the products. 
 The metadata lives either in a Kuva Space database, or alternatively in a sidecar JSON file. 
 
-This library allows the reading of the image GeoTIFFs into `xarray.Dataset` objects that 
+This library allows the reading of the image GeoTIFFs into `rasterio.DatasetReader` objects that 
 allow convenient raster manipulations, along with their `kuva-metadata` metadata objects.
 
 # Installation
@@ -33,7 +33,7 @@ pip install kuva-reader
 This is a minimal example that allows you to read and print the image shape of a L2 product.
 
 The result product is in this case an L2A product (as seen from the folder name).
-The loaded product is stored in a `rioxarray` object, which contains extensive GIS functionalities [(examples for usage)](https://corteva.github.io/rioxarray/stable/examples/examples.html).
+The loaded product is stored in a `rasterio.DatasetReader` object, which contains extensive GIS functionalities [(examples for usage)](https://rasterio.readthedocs.io/en/stable/api/rasterio.io.html#rasterio.io.DatasetReader).
 
 ```python
 from kuva_reader import read_product

{kuva_reader-1.0.4 → kuva_reader-1.1.6}/kuva_reader/__init__.py

@@ -17,17 +17,14 @@ Key Features
 Dependencies
 - kuva-metadata: A specialized library that handles the extraction and
   parsing of metadata associated with Kuva Space products.
-- xarray: Used for loading image data as arrays with extra functionality,
-  including labeled coordinates and metadata, which is useful for analysis and
+- rasterio: Used for loading image data as arrays with extra functionality,
+  including GIS specific functions and metadata, which are useful for analysis and
   visualization.
 """
 
-__version__ = "0.1.0"
+__version__ = "1.1.2"
 
 from .reader.image import (
-    image_to_dtype_range,
-    image_to_original_range,
-    image_to_uint16_range,
     image_footprint,
 )
 from .reader.level0 import Level0Product
@@ -40,9 +37,6 @@ __all__ = [
     "Level1ABProduct",
     "Level1CProduct",
     "Level2AProduct",
-    "image_to_dtype_range",
-    "image_to_original_range",
-    "image_to_uint16_range",
     "image_footprint",
     "read_product",
 ]

kuva_reader-1.1.6/kuva_reader/reader/image.py

@@ -0,0 +1,28 @@
+"""Utilities to process images related to product processing."""
+
+import rasterio as rio
+from shapely.geometry import box, Polygon
+from rasterio.warp import transform_bounds
+
+
+def image_footprint(image: rio.DatasetReader, crs: str = "") -> Polygon:
+    """Return a product footprint as a shapely polygon
+
+    Parameters
+    ----------
+    image
+        The product image
+    crs, optional
+        CRS to convert to, by default "", keeping the image's CRS
+
+    Returns
+    -------
+        A shapely polygon footprint
+    """
+    if crs:
+        # Transform the bounds to the new CRS using rasterio's built-in function
+        bounds = transform_bounds(image.crs, crs, *image.bounds)
+        footprint = box(*bounds)
+    else:
+        footprint = box(*image.bounds)
+    return footprint

{kuva_reader-1.0.4 → kuva_reader-1.1.6}/kuva_reader/reader/level0.py

@@ -2,15 +2,14 @@ from pathlib import Path
 from typing import cast
 
 import numpy as np
-import rioxarray as rx
-import xarray
+import rasterio as rio
 from kuva_metadata import MetadataLevel0
 from pint import UnitRegistry
 from shapely import Polygon
 
-from kuva_reader import image_to_dtype_range, image_to_original_range, image_footprint
+from kuva_reader import image_footprint
 
-from .product_base import ProductBase
+from .product_base import NUM_THREADS, ProductBase
 
 
 class Level0Product(ProductBase[MetadataLevel0]):
@@ -39,13 +38,6 @@ class Level0Product(ProductBase[MetadataLevel0]):
     target_ureg, optional
         Pint Unit Registry to swap to. This is only relevant when parsing data from a
         JSON file, which by default uses the kuva-metadata ureg.
-    as_physical_unit
-        Whether to denormalize data from full data type range back to the physical
-        units stored with the data, by default False
-    target_dtype
-        Target data type to normalize data to. This will first denormalize the data
-        to its original range and then normalize to new data type range to keep a
-        scale and offset, by default None
 
     Attributes
     ----------
@@ -53,10 +45,9 @@ class Level0Product(ProductBase[MetadataLevel0]):
         Path to the folder containing the images.
     metadata: MetadataLevel0
         The metadata associated with the images
-    images: Dict[str, xarray.DataArray]
-        The arrays with the actual data. This have the rioxarray extension activated on
-        them so lots of GIS functionality are available on them. Imporantly, the GCPs
-        can be retrieved like so: `ds.rio.get_gcps()`
+    images: Dict[str, rasterio.DatasetReader]
+        A dictionary that maps camera names to their respective Rasterio DatasetReader
+        objects.
     data_tags: Dict[str, Any]
         Tags stored along with the data. These can be used e.g. to check the physical
         units of pixels or normalisation factors.
@@ -67,58 +58,53 @@ class Level0Product(ProductBase[MetadataLevel0]):
         image_path: Path,
         metadata: MetadataLevel0 | None = None,
         target_ureg: UnitRegistry | None = None,
-        as_physical_unit: bool = False,
-        target_dtype: np.dtype | None = None,
     ) -> None:
         super().__init__(image_path, metadata, target_ureg)
 
-        self.images = {
+        self._images = {
             camera: cast(
-                xarray.DataArray,
-                rx.open_rasterio(
+                rio.DatasetReader,
+                rio.open(
                     self.image_path / (cube.camera.name + ".tif"),
+                    num_threads=NUM_THREADS,
                 ),
             )
             for camera, cube in self.metadata.image.data_cubes.items()  # type: ignore
         }
-        self.crs = self.images[list(self.images.keys())[0]].rio.crs
+        self.crs = self.images[list(self.images.keys())[0]].crs
 
         # Read tags for images and denormalize / renormalize if needed
-        self.data_tags = {camera: img.attrs for camera, img in self.images.items()}
-        if as_physical_unit or target_dtype:
-            for camera, img in self.images.items():
-                # Move from normalized full scale back to original data float values.
-                # pop() since values not true anymore after denormalization.
-                norm_img = image_to_original_range(
-                    img,
-                    self.data_tags[camera].pop("data_offset"),
-                    self.data_tags[camera].pop("data_scale"),
-                )
-                self.images[camera] = norm_img
-
-                if target_dtype:
-                    # For algorithm needs, cast and normalize to a specific dtype range
-                    # NOTE: This may remove data precision e.g. uint16 -> uint8
-                    norm_img, offset, scale = image_to_dtype_range(img, target_dtype)
-                    self.data_tags[camera]["data_offset"] = offset
-                    self.data_tags[camera]["data_scale"] = scale
+        self.data_tags = {camera: src.tags() for camera, src in self.images.items()}
 
     def __repr__(self):
         """Pretty printing of the object with the most important info"""
         if self.images is not None and len(self.images):
+            image_shapes = []
+            for camera_name, image in self.images.items():
+                shape_str = f"({image.count}, {image.height}, {image.width})"
+                image_shapes.append(f"{camera_name.upper()} shape {shape_str}")
+
+            shapes_description = " and ".join(image_shapes)
+
             return (
-                f"{self.__class__.__name__}"
-                f"with VIS shape {self.images['vis'].shape} "
-                f"and NIR shape {self.images['nir'].shape} "
-                f"(CRS '{self.crs}'). Loaded from: '{self.image_path}'."
+                f"{self.__class__.__name__} "
+                f"with {shapes_description} and "
+                f"CRS: '{self.crs}'. Loaded from: '{self.image_path}'."
             )
         else:
             return f"{self.__class__.__name__} loaded from '{self.image_path}'."
 
-    def __getitem__(self, camera: str) -> xarray.DataArray:
+    def __getitem__(self, camera: str) -> rio.DatasetReader:
         """Return the datarray for the chosen camera."""
         return self.images[camera]
 
+    @property
+    def images(self) -> dict[str, rio.DatasetReader]:
+        if self._images is None:
+            e_ = "Images have been released. Re-open the product to access it again."
+            raise RuntimeError(e_)
+        return self._images
+
     def keys(self) -> list[str]:
         """Easy access to the camera keys."""
         return list(self.images.keys())
@@ -192,7 +178,11 @@ class Level0Product(ProductBase[MetadataLevel0]):
     def read_frame(self, cube: str, band_id: int, frame_idx: int) -> np.ndarray:
         """Extract a specific frame from a cube and band."""
         frame_offset = self.calculate_frame_offset(cube, band_id, frame_idx)
-        return self[cube][frame_offset, :, :].to_numpy()
+
+        # Rasterio index starts at 1
+        frame_offset += 1
+
+        return self[cube].read(frame_offset)
 
     def read_band(self, cube: str, band_id: int) -> np.ndarray:
         """Extract a specific band from a cube"""
@@ -201,7 +191,12 @@ class Level0Product(ProductBase[MetadataLevel0]):
         # Calculate the final frame offset for this band and frame
         band_offset_ll = band_offsets[band_id]
         band_offset_ul = band_offset_ll + band_n_frames[band_id]
-        return self[cube][band_offset_ll:band_offset_ul, :, :].to_numpy()
+
+        # Rasterio index starts at 1
+        band_offset_ll += 1
+        band_offset_ul += 1
+
+        return self[cube].read(list(np.arange(band_offset_ll, band_offset_ul)))
 
     def read_data_units(self) -> np.ndarray:
         """Read unit of product and validate they match between cameras"""
@@ -213,7 +208,7 @@ class Level0Product(ProductBase[MetadataLevel0]):
             e_ = "Cameras have different physical units stored to them."
             raise ValueError(e_)
 
-    def get_bad_pixel_mask(self, camera: str | None = None) -> xarray.Dataset:
+    def get_bad_pixel_mask(self, camera: str | None = None) -> rio.DatasetReader:
         """Get the bad pixel mask associated to each camera of the L0 product
 
         Returns
@@ -226,7 +221,7 @@ class Level0Product(ProductBase[MetadataLevel0]):
         bad_pixel_filename = f"{camera}_per_frame_bad_pixel_mask.tif"
         return self._read_array(self.image_path / bad_pixel_filename)
 
-    def get_cloud_mask(self, camera: str | None = None) -> xarray.Dataset:
+    def get_cloud_mask(self, camera: str | None = None) -> rio.DatasetReader:
         """Get the cloud mask associated to the product.
 
         Returns
@@ -240,14 +235,17 @@ class Level0Product(ProductBase[MetadataLevel0]):
         return self._read_array(self.image_path / bad_pixel_filename)
 
     def release_memory(self):
-        """Explicitely releases the memory of the `images` variable.
-
-        NOTE: this function is implemented because of a memory leak inside the Rioxarray
-        library that doesn't release memory properly. Only use it when the image data is
-        not needed anymore.
+        """Explicitely closes the Rasterio DatasetReaders and releases the memory of
+        the `images` variable.
         """
-        del self.images
-        self.images = None
+        if self._images is not None:
+            for k in self._images.keys():
+                self._images[k].close()
+
+            del self._images
+            # We know that images are not None as long as somebody doesn't call
+            # this function beforehand....
+            self._images = None
 
 
 def generate_level_0_metafile():