simcats-datasets 2.4.0__tar.gz → 2.6.0__tar.gz

simcats_datasets-2.6.0/PKG-INFO

@@ -0,0 +1,163 @@
+Metadata-Version: 2.4
+Name: simcats-datasets
+Version: 2.6.0
+Summary: SimCATS-Datasets is a Python package that simplifies the creation and loading of SimCATS datasets. 
+Author-email: Fabian Hader <f.hader@fz-juelich.de>, Fabian Fuchs <f.fuchs@fz-juelich.de>, Karin Havemann <k.havemann@fz-juelich.de>, Sarah Fleitmann <s.fleitmann@fz-juelich.de>, Jan Vogelbruch <j.vogelbruch@fz-juelich.de>
+License-Expression: GPL-3.0-or-later
+Project-URL: homepage, https://github.com/f-hader/SimCATS-Datasets
+Project-URL: documentation, https://simcats-datasets.readthedocs.io
+Project-URL: source, https://github.com/f-hader/SimCATS-Datasets
+Project-URL: tracker, https://github.com/f-hader/SimCATS-Datasets/issues
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering
+Classifier: Typing :: Typed
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: bezier
+Requires-Dist: bm3d
+Requires-Dist: h5py
+Requires-Dist: hdf5storage
+Requires-Dist: numpy
+Requires-Dist: opencv-python
+Requires-Dist: pandas
+Requires-Dist: parallelbar
+Requires-Dist: parse
+Requires-Dist: scikit-image
+Requires-Dist: simcats>=2.0.0
+Requires-Dist: torch
+Requires-Dist: tqdm
+Requires-Dist: xarray
+Dynamic: license-file
+
+<h1 align="center">
+  <img src="https://raw.githubusercontent.com/f-hader/SimCATS-Datasets/main/SimCATS-Datasets_symbol.svg" alt="SimCATS logo">
+  <br>
+</h1>
+
+<div align="center">
+  <a href="https://github.com/f-hader/SimCATS-Datasets/blob/main/LICENSE">
+    <img src="https://img.shields.io/badge/License-GPLv3-blue.svg" alt="License: GPLv3"/>
+  </a>
+  <a href="https://pypi.org/project/simcats-datasets/">
+    <img src="https://img.shields.io/pypi/v/simcats-datasets.svg" alt="PyPi Latest Release"/>
+  </a>
+  <a href="https://simcats-datasets.readthedocs.io/en/latest/">
+    <img src="https://img.shields.io/readthedocs/simcats-datasets" alt="Read the Docs"/>
+  </a>
+  <a href="https://doi.org/10.1109/TQE.2024.3445967">
+    <img src="https://img.shields.io/badge/DOI (SimCATS Paper)-10.1109/TQE.2024.3445967-007ec6.svg" alt="DOI Paper"/>
+  </a>
+  <a href="https://doi.org/10.5281/zenodo.13862231">
+    <img src="https://img.shields.io/badge/DOI (Code)-10.5281/zenodo.13862231-007ec6.svg" alt="DOI Code"/>
+  </a>
+</div>
+
+# SimCATS-Datasets
+
+`SimCATS-Datasets` is a Python package that simplifies the creation and loading of `SimCATS` datasets. Please have a look at 
+[this repository](https://github.com/f-hader/SimCATS) regarding `SimCATS` itself.
+
+## Installation
+
+The framework supports Python versions 3.7 - 3.11 and installs via pip:
+```
+pip install simcats-datasets
+```
+
+Alternatively, the `SimCATS-Datasets` package can be installed by cloning the GitHub repository, navigating to the
+folder containing the `setup.py` file, and executing
+```
+pip install .
+```
+
+For installation in development/editable mode, use the option `-e`.
+
+<!-- start sec:documentation -->
+## Documentation
+
+The official documentation is hosted on [ReadtheDocs](https://simcats-datasets.readthedocs.io) but can also be built
+locally. To do this, first install the packages `sphinx`, `sphinx-rtd-theme`, `sphinx-autoapi`, `myst-nb `, and 
+`jupytext` with
+
+```
+pip install sphinx sphinx-rtd-theme sphinx-autoapi myst-nb jupytext
+```
+
+and then, in the `docs` folder, execute the following command:
+
+```
+.\make html
+```
+
+To view the generated HTML documentation, open the file `docs\build\html\index.html`.
+<!-- end sec:documentation -->
+
+
+## Loading Datasets
+
+Datasets created with `SimCATS-Datasets` are stored in HDF5 files. These datasets can be loaded using the function 
+`load_dataset` from `simcats_datasets.loading`.
+
+The return value of the function is a named tuple. The fields can be accessed by their name or index. As with normal
+tuples, it is also possible to unpack the returned fields directly into separate variables. The available fields 
+depend on which data was specified to be loaded. Please look at the docstring for further information.
+
+Additionally, `SimCATS-Datasets` offers a pytorch dataset (see `torch.utils.data.Dataset`) implementation called 
+`SimcatsDataset`. It allows the direct use of `SimCATS` datasets for machine learning purposes with Torch and can be 
+imported from `simcats_datasets.loading.pytorch`.
+
+## Creating Datasets
+
+To create a simulated dataset, import `create_simulated_dataset` from `simcats_datasets.generation`. This function
+allows the creation of simulated CSDs with ground truth very easily. It is also possible to add further CSDs to already
+existing datasets. The function will detect the existing dataset automatically. For the function's usage, please have a
+look at its docstring. 
+
+| :warning: WARNING                                                                                                                                                                                                                                                                                                        |
+|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| The functionalities for creating and extending simulated datasets using SimCATS expect that the SimCATS simulation uses the IdealCSDInterface implementation called IdealCSDGeometric. Other implementations might cause problems because the expected information for creating labeled lines etc. might be unavailable. |  
+
+
+Alternatively, to using `create_simulated_dataset` and directly simulating a dataset with `SimCATS`, it is also possible 
+to create a `SimCATS-Dataset` compatible dataset with existing data (for example, experimentally measured data or data 
+simulated with other frameworks). This can be done using `create_dataset` from `simcats_datasets.generation`.
+
+## Citations
+
+```bibtex
+@article{hader2024simcats,
+  author={Hader, Fabian and Fleitmann, Sarah and Vogelbruch, Jan and Geck, Lotte and Waasen, Stefan van},
+  journal={IEEE Transactions on Quantum Engineering}, 
+  title={Simulation of Charge Stability Diagrams for Automated Tuning Solutions (SimCATS)}, 
+  year={2024},
+  volume={5},
+  pages={1-14},
+  doi={10.1109/TQE.2024.3445967}
+}
+```
+
+## License, CLA, and Copyright
+
+[![CC BY-NC-SA 4.0][gplv3-shield]][gplv3]
+
+This work is licensed under a
+[GNU General Public License 3][gplv3].
+
+[![GPLv3][gplv3-image]][gplv3]
+
+[gplv3]: https://www.gnu.org/licenses/gpl-3.0.html
+[gplv3-image]: https://www.gnu.org/graphics/gplv3-127x51.png
+[gplv3-shield]: https://img.shields.io/badge/License-GPLv3-blue.svg
+
+Contributions must follow the Contributor License Agreement. For more information, see the [CONTRIBUTING.md](https://github.com/f-hader/SimCATS-Datasets/blob/main/CONTRIBUTING.md) file at the top of the GitHub repository.
+
+Copyright © 2026 Peter Grünberg Institute - Integrated Computing Architectures (ICA / PGI-4), Forschungszentrum Jülich GmbH

{simcats_datasets-2.4.0 → simcats_datasets-2.6.0}/README.md

@@ -120,4 +120,4 @@ This work is licensed under a
 
 Contributions must follow the Contributor License Agreement. For more information, see the [CONTRIBUTING.md](https://github.com/f-hader/SimCATS-Datasets/blob/main/CONTRIBUTING.md) file at the top of the GitHub repository.
 
-Copyright © 2024 Forschungszentrum Jülich GmbH - Central Institute of Engineering, Electronics and Analytics (ZEA) - Electronic Systems (ZEA-2)
+Copyright © 2026 Peter Grünberg Institute - Integrated Computing Architectures (ICA / PGI-4), Forschungszentrum Jülich GmbH

{simcats_datasets-2.4.0 → simcats_datasets-2.6.0}/pyproject.toml

@@ -1,11 +1,12 @@
 [build-system]
-requires = ["setuptools>=61.0"]
+requires = ["setuptools>=77.0"]
 build-backend = "setuptools.build_meta"
 
 [project]
 name = "simcats-datasets"
-version = "2.4.0" # also change in docs/source/conf.py and __init__
-license = { file="LICENSE" }
+version = "2.6.0" # also change in docs/source/conf.py and __init__
+license = "GPL-3.0-or-later"
+license-files = ["LICENSE"]
 authors = [
     { name="Fabian Hader", email="f.hader@fz-juelich.de" },
     { name="Fabian Fuchs", email="f.fuchs@fz-juelich.de" },
@@ -19,7 +20,6 @@ description = """\
 readme = "README.md"
 requires-python = ">=3.7"
 classifiers = [
-    'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
     'Development Status :: 5 - Production/Stable',
     'Intended Audience :: Science/Research',
     'Programming Language :: Python',
@@ -43,7 +43,7 @@ dependencies = [
     "parallelbar",
     "parse",
     "scikit-image",
-    "simcats>=1.2.0",
+    "simcats>=2.0.0",
     "torch",
     "tqdm",
     "xarray"

simcats_datasets-2.6.0/simcats_datasets/__init__.py

@@ -0,0 +1,2 @@
+__all__ = []
+__version__ = "2.6.0"

{simcats_datasets-2.4.0 → simcats_datasets-2.6.0}/simcats_datasets/generation/_create_dataset.py

@@ -16,10 +16,13 @@ __all__ = []
 
 
 def create_dataset(dataset_path: str,
-                   csds: List[np.ndarray],
+                   csds: Optional[List[np.ndarray]] = None,
+                   sensor_scans: Optional[List[np.ndarray]] = None,
                    occupations: Optional[List[np.ndarray]] = None,
                    tct_masks: Optional[List[np.ndarray]] = None,
                    ct_by_dot_masks: Optional[List[np.ndarray]] = None,
+                   sensor_regime_masks: Optional[List[np.ndarray]] = None,
+                   sensor_peak_center_masks: Optional[List[np.ndarray]] = None,
                    line_coordinates: Optional[List[np.ndarray]] = None,
                    line_labels: Optional[List[dict]] = None,
                    metadata: Optional[List[dict]] = None,
@@ -27,60 +30,103 @@ def create_dataset(dataset_path: str,
                    max_len_line_labels_chunk: Optional[int] = None,
                    max_len_metadata_chunk: Optional[int] = None,
                    dtype_csd: np.dtype = np.float32,
+                   dtype_sensor_scan: np.dtype = np.float32,
                    dtype_occ: np.dtype = np.float32,
                    dtype_tct: np.dtype = np.uint8,
                    dtype_ct_by_dot: np.dtype = np.uint8,
+                   dtype_sensor_regime_masks: np.dtype = np.uint8,
+                   dtype_sensor_peak_center_masks: np.dtype = np.uint8,
                    dtype_line_coordinates: np.dtype = np.float32) -> None:
     """Function for creating simcats_datasets v2 format datasets from given data.
 
     Args:
         dataset_path: The path where the new (v2) HDF5 dataset will be stored.
-        csds: The list of CSDs to use for creating the dataset.
+        csds: The list of CSDs to use for creating the dataset. A dataset can have either CSDs or sensor scans, but
+            never both. Default is None.
+        sensor_scans: The list of sensor scans to use for creating the dataset. A dataset can have either CSDs or sensor
+            scans, but never both. Default is None.
         occupations: List of occupations to use for creating the dataset. Defaults to None.
         tct_masks: List of TCT masks to use for creating the dataset. Defaults to None.
         ct_by_dot_masks: List of CT by dot masks to use for creating the dataset. Defaults to None.
+        sensor_regime_masks: List of sensor regime masks to use for creating the dataset. Defaults to None.
+        sensor_peak_center_masks: List of sensor peak center masks to use for creating the dataset. Defaults to None.
         line_coordinates: List of line coordinates to use for creating the dataset. Defaults to None.
         line_labels: List of line labels to use for creating the dataset. Defaults to None.
         metadata: List of metadata to use for creating the dataset. Defaults to None.
         max_len_line_coordinates_chunk: The expected maximal length for line coordinates in number of float values (each
-            line requires 4 floats). If None, it is set to the largest value of the CSD shape. Default is None.
+            line requires 4 floats). If None, it is set to the largest value of the CSD (or sensor scan) shape. Default
+            is None.
         max_len_line_labels_chunk: The expected maximal length for line labels in number of uint8/char values (each line
             label, encoded as utf-8 json, should require at most 80 chars). If None, it is set to the largest value of
-            the CSD shape * 20 (matching with allowed number of line coords). Default is None.
+            the CSD (or sensor scan) shape * 20 (matching with allowed number of line coords). Default is None.
         max_len_metadata_chunk: The expected maximal length for metadata in number of uint8/char values (each metadata
             dict, encoded as utf-8 json, should require at most 8000 chars, expected rather something like 4000, but
             could get larger for dot jumps metadata of high resolution scans). If None, it is set to 8000. Default is
             None.
         dtype_csd: Specifies the dtype to be used for saving CSDs. Default is np.float32.
+        dtype_sensor_scan: Specifies the dtype to be used for saving sensor scans. Default is np.float32.
         dtype_occ: Specifies the dtype to be used for saving Occupations. Default is np.float32.
         dtype_tct: Specifies the dtype to be used for saving TCTs. Default is np.uint8.
         dtype_ct_by_dot: Specifies the dtype to be used for saving CT by dot masks. Default is np.uint8.
+        dtype_sensor_regime_masks: Specifies the dtype to be used for saving sensor regime masks. Default is np.uint8.
+        dtype_sensor_peak_center_masks: Specifies the dtype to be used for saving sensor peak center masks. Default is
+            np.uint8.
         dtype_line_coordinates: Specifies the dtype to be used for saving line coordinates. Default is np.float32.
     """
     # Create path where the dataset will be saved (if folder doesn't exist already)
     Path(dirname(dataset_path)).mkdir(parents=True, exist_ok=True)
 
+    # check if the dataset to be created is a csd or sensor_scan dataset
+    if csds is not None and sensor_scans is None:
+        csd_dataset = True
+    elif csds is None and sensor_scans is not None:
+        csd_dataset = False
+    else:
+        raise ValueError("A dataset can contain either CSDs or sensor scans but never both! Exactly one of the two has "
+                         "to be None.")
+
     with h5py.File(dataset_path, "a") as hdf5_file:
         # get the number of total ids. This is especially required if a large dataset is loaded and saved step by step
-        num_ids = len(csds)
+        if csd_dataset:
+            num_ids = len(csds)
+        else:
+            num_ids = len(sensor_scans)
 
-        # process CSDs
-        # save an example CSD to get shape and dtype
-        temp_csd = csds[0].copy()
-        # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to load
-        # one image at a time during training)
-        ds = hdf5_file.require_dataset(name='csds', shape=(0, *temp_csd.shape), dtype=dtype_csd,
-                                       maxshape=(None, *temp_csd.shape))
+        # get a temp copy of a csd or sensor scan (to get the shape) and retrieve the corresponding HDF5 dataset
+        if csd_dataset:
+            # process CSDs
+            # save an example CSD to get shape and dtype
+            temp_data = csds[0].copy()
+            # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
+            # load one image at a time during training)
+            ds = hdf5_file.require_dataset(name='csds',
+                                           shape=(0, *temp_data.shape),
+                                           dtype=dtype_csd,
+                                           maxshape=(None, *temp_data.shape))
+        else:
+            # process sensor scans
+            # save an example sensor scan to get shape and dtype
+            temp_data = sensor_scans[0].copy()
+            # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
+            # load one image at a time during training)
+            ds = hdf5_file.require_dataset(name='sensor_scans',
+                                           shape=(0, *temp_data.shape),
+                                           dtype=dtype_sensor_scan,
+                                           maxshape=(None, *temp_data.shape))
         # determine index offset if there is already data in the dataset
         id_offset = ds.shape[0]
         # resize datasets to fit new data
         ds.resize(ds.shape[0] + num_ids, axis=0)
-        ds[id_offset:] = np.array(csds).astype(dtype_csd)
+        # Add new CSDs or sensor scans to the dataset
+        if csd_dataset:
+            ds[id_offset:] = np.array(csds).astype(dtype_csd)
+        else:
+            ds[id_offset:] = np.array(sensor_scans).astype(dtype_sensor_scan)
         if occupations is not None:
             if len(occupations) != num_ids:
                 raise ValueError(
-                    f"Number of new occupation arrays ({len(occupations)}) does not match the number of new CSDs "
-                    f"({num_ids}).")
+                    f"Number of new occupation arrays ({len(occupations)}) does not match the number of new CSDs or "
+                    f"sensor scans ({num_ids}).")
             # process Occupations
             # save an example occ to get shape
             temp_occ = occupations[0].copy()
@@ -91,15 +137,15 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored occupation arrays ({ds.shape[0]}) does not match the number of already "
-                    f"stored CSDs ({id_offset}).")
+                    f"stored CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             ds[id_offset:] = np.array(occupations).astype(dtype_occ)
         if tct_masks is not None:
             if len(tct_masks) != num_ids:
                 raise ValueError(
-                    f"Number of new TCT mask arrays ({len(tct_masks)}) does not match the number of new CSDs "
-                    f"({num_ids}).")
+                    f"Number of new TCT mask arrays ({len(tct_masks)}) does not match the number of new CSDs or sensor "
+                    f"scans ({num_ids}).")
             # process tct masks
             # save an example tct to get shape and dtype
             temp_tct = tct_masks[0].copy()
@@ -110,7 +156,7 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored TCT mask arrays ({ds.shape[0]}) does not match the number of already "
-                    f"stored CSDs ({id_offset}).")
+                    f"stored CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             ds[id_offset:] = np.array(tct_masks).astype(dtype_tct)
@@ -118,7 +164,7 @@ def create_dataset(dataset_path: str,
             if len(ct_by_dot_masks) != num_ids:
                 raise ValueError(
                     f"Number of new CT by dot mask arrays ({len(ct_by_dot_masks)}) does not match the number of new "
-                    f"CSDs ({num_ids}).")
+                    f"CSDs or sensor scans ({num_ids}).")
             # process tct masks
             # save an example tct to get shape and dtype
             temp_ct_by_dot = ct_by_dot_masks[0].copy()
@@ -129,19 +175,60 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored CT by dot mask arrays ({ds.shape[0]}) does not match the number of "
-                    f"already stored CSDs ({id_offset}).")
+                    f"already stored CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             ds[id_offset:] = np.array(ct_by_dot_masks).astype(dtype_tct)
+        if sensor_regime_masks is not None:
+            if len(sensor_regime_masks) != num_ids:
+                raise ValueError(
+                    f"Number of new sensor regime mask arrays ({len(sensor_regime_masks)}) does not match the number "
+                    f"of new CSDs or sensor scans ({num_ids}).")
+            # process sensor regime masks
+            # save an example sensor regime mask to get shape and dtype
+            temp_sensor_regime_mask = sensor_regime_masks[0].copy()
+            # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
+            # load one image at a time during training)
+            ds = hdf5_file.require_dataset(name='sensor_regime_masks', shape=(0, *temp_sensor_regime_mask.shape),
+                                           dtype=dtype_sensor_regime_masks,
+                                           maxshape=(None, *temp_sensor_regime_mask.shape))
+            if ds.shape[0] != id_offset:
+                raise ValueError(
+                    f"Number of already stored sensor regime mask arrays ({ds.shape[0]}) does not match the number of "
+                    f"already stored CSDs or sensor scans ({id_offset}).")
+            # resize datasets to fit new data
+            ds.resize(ds.shape[0] + num_ids, axis=0)
+            ds[id_offset:] = np.array(sensor_regime_masks).astype(dtype_sensor_regime_masks)
+        if sensor_peak_center_masks is not None:
+            if len(sensor_peak_center_masks) != num_ids:
+                raise ValueError(
+                    f"Number of new sensor peak center mask arrays ({len(sensor_peak_center_masks)}) does not match "
+                    f"the number of new CSDs or sensor scans ({num_ids}).")
+            # process sensor peak center masks
+            # save an example sensor peak center mask to get shape and dtype
+            temp_sensor_peak_center_mask = sensor_peak_center_masks[0].copy()
+            # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
+            # load one image at a time during training)
+            ds = hdf5_file.require_dataset(name='sensor_peak_center_masks',
+                                           shape=(0, *temp_sensor_peak_center_mask.shape),
+                                           dtype=dtype_sensor_peak_center_masks,
+                                           maxshape=(None, *temp_sensor_peak_center_mask.shape))
+            if ds.shape[0] != id_offset:
+                raise ValueError(
+                    f"Number of already stored sensor peak center mask arrays ({ds.shape[0]}) does not match the "
+                    f"number of already stored CSDs or sensor scans ({id_offset}).")
+            # resize datasets to fit new data
+            ds.resize(ds.shape[0] + num_ids, axis=0)
+            ds[id_offset:] = np.array(sensor_peak_center_masks).astype(dtype_sensor_peak_center_masks)
         if line_coordinates is not None:
             if len(line_coordinates) != num_ids:
                 raise ValueError(
                     f"Number of new line coordinates ({len(line_coordinates)}) does not match the number of new "
-                    f"CSDs ({num_ids}).")
+                    f"CSDs or sensor scans ({num_ids}).")
             # retrieve fixed length for chunks
             if max_len_line_coordinates_chunk is None:
                 # calculate max expected length (max_number_of_lines * 4 entries, max number estimated as max(shape)/4)
-                max_len = max(temp_csd.shape)
+                max_len = max(temp_data.shape)
             else:
                 max_len = max_len_line_coordinates_chunk
             # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
@@ -151,7 +238,7 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored line coordinates ({ds.shape[0]}) does not match the number of already "
-                    f"stored CSDs ({id_offset}).")
+                    f"stored CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             # process line coordinates
@@ -163,13 +250,13 @@ def create_dataset(dataset_path: str,
         if line_labels is not None:
             if len(line_labels) != num_ids:
                 raise ValueError(
-                    f"Number of new line labels ({len(line_labels)}) does not match the number of new CSDs "
-                    f"({num_ids}).")
+                    f"Number of new line labels ({len(line_labels)}) does not match the number of new CSDs or sensor "
+                    f"scans ({num_ids}).")
             # retrieve fixed length for chunks
             if max_len_line_labels_chunk is None:
                 # calculate max expected length (max_number_of_lines * 80 uint8 numbers, max number estimated as
                 # max(shape)/4)
-                max_len = max(temp_csd.shape) * 20
+                max_len = max(temp_data.shape) * 20
             else:
                 max_len = max_len_line_labels_chunk
             # use chunks as this will speed up reading later! One chunk is set to be exactly one image (optimized to
@@ -179,7 +266,7 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored line labels ({ds.shape[0]}) does not match the number of already stored "
-                    f"CSDs ({id_offset}).")
+                    f"CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             # process line labels
@@ -193,7 +280,8 @@ def create_dataset(dataset_path: str,
         if metadata is not None:
             if len(metadata) != num_ids:
                 raise ValueError(
-                    f"Number of new metadata ({len(metadata)}) does not match the number of new CSDs ({num_ids}).")
+                    f"Number of new metadata ({len(metadata)}) does not match the number of new CSDs or sensor scans "
+                    f"({num_ids}).")
             # retrieve fixed length for chunks
             if max_len_metadata_chunk is None:
                 # set len to 8000 uint8 numbers, that should already include some extra safety (expected smth. like
@@ -208,7 +296,7 @@ def create_dataset(dataset_path: str,
             if ds.shape[0] != id_offset:
                 raise ValueError(
                     f"Number of already stored metadata ({ds.shape[0]}) does not match the number of already stored "
-                    f"CSDs ({id_offset}).")
+                    f"CSDs or sensor scans ({id_offset}).")
             # resize datasets to fit new data
             ds.resize(ds.shape[0] + num_ids, axis=0)
             # process metadata