das2numpy 1.0__tar.gz → 1.1__tar.gz

das2numpy-1.1/PKG-INFO

@@ -0,0 +1,102 @@
+Metadata-Version: 2.4
+Name: das2numpy
+Version: 1.1
+Summary: A simple and universal package for loading large amounts of distributed acoustic sensing (DAS) data.
+Author-email: Erik Genthe <erik.genthe@desy.de>
+Project-URL: Homepage, https://git.physnet.uni-hamburg.de/wave/das2numpy
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: ffmpeg-python
+Requires-Dist: h5py
+Requires-Dist: scipy
+Requires-Dist: numba
+Dynamic: license-file
+
+# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
+
+
+
+## Install
+
+You can install via PIP.
+```
+python -m pip install das2numpy
+```
+
+To load data from flac files, ffmpeg (https://ffmpeg.org) needs to be installed. It is not possible to install ffmpeg with pip.
+
+On DESY's Maxwell cluster ffmpeg is available as a module. Before using das2numpy execute:
+```
+module load maxwell ffmpeg
+```
+
+
+
+
+## Python API
+
+Example: If you want to get started quickly, have a look at the [example.py](src/example.py).
+
+Create an instance with:
+
+```python
+def loader(root_path:str, predefined_setup:str, num_worker_threads):
+```
+```
+    Loads data and returns it as a numpy array. 
+    Args:
+        root_path (str): Path to directory that contains the files to be loaded from. Subdirectories are (recursively) also searched.
+        predefined_setup (str): One of ["SILIXA", "FLAC_200HZ", "OPTASENSE"]
+        num_worker_threads (int): The number of worker threads used for loading files in parallel.
+    Returns:
+        A loader instance to load data. Call instance.load_array(...).
+```
+
+Use one of the load_array(..) functions of that instance.
+
+```python
+def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
+```
+```
+Loading data into numpy array.
+Returns nothing, the data can be accessed by accessing the data field of this instance.
+Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
+        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
+Args:
+    t_start (datetime): datetime object which defines the start of the data to load.
+    t_end (datetime): datetime object which defines the end of the data to load.
+    channel_start (int): The starting index of the sensor position in the data (inclusive).
+    channel_end (int): The ending index of the sensors position in the data (exclusive).
+    t_step (int): Reduces the data on the time axis by factor t_step. Uses mean averaging. Default is 1. 
+    channel_step (int): Like t_step, but for the sensor position.
+Returns:
+    A 2d-numpy-array containing the data.
+    The first axis corresponds to the time, the second to the channel (sensor position)
+ ```
+
+For more details have a look at the inline documentation of [chunk.py](src/das2numpy/chunk.py)
+
+
+## Command Line Interface
+
+Creates a numpy file from the requested data. Optionally, the binary data can be printed to stdout.
+
+Example call:
+```
+python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2024-DESY/2024-07-23-desy 2024-07-23T10:01:00 2024-07-23T10:02:00 10 0 1000 10 default
+```
+
+For more information:
+```
+python -m das2numpy -h
+```
+
+
+## Issues
+
+- Loading from OPTASENSE may not work anymore. I haven't tested it for a long time.

das2numpy-1.1/README.md

@@ -0,0 +1,83 @@
+# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
+
+
+
+## Install
+
+You can install via PIP.
+```
+python -m pip install das2numpy
+```
+
+To load data from flac files, ffmpeg (https://ffmpeg.org) needs to be installed. It is not possible to install ffmpeg with pip.
+
+On DESY's Maxwell cluster ffmpeg is available as a module. Before using das2numpy execute:
+```
+module load maxwell ffmpeg
+```
+
+
+
+
+## Python API
+
+Example: If you want to get started quickly, have a look at the [example.py](src/example.py).
+
+Create an instance with:
+
+```python
+def loader(root_path:str, predefined_setup:str, num_worker_threads):
+```
+```
+    Loads data and returns it as a numpy array. 
+    Args:
+        root_path (str): Path to directory that contains the files to be loaded from. Subdirectories are (recursively) also searched.
+        predefined_setup (str): One of ["SILIXA", "FLAC_200HZ", "OPTASENSE"]
+        num_worker_threads (int): The number of worker threads used for loading files in parallel.
+    Returns:
+        A loader instance to load data. Call instance.load_array(...).
+```
+
+Use one of the load_array(..) functions of that instance.
+
+```python
+def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
+```
+```
+Loading data into numpy array.
+Returns nothing, the data can be accessed by accessing the data field of this instance.
+Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
+        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
+Args:
+    t_start (datetime): datetime object which defines the start of the data to load.
+    t_end (datetime): datetime object which defines the end of the data to load.
+    channel_start (int): The starting index of the sensor position in the data (inclusive).
+    channel_end (int): The ending index of the sensors position in the data (exclusive).
+    t_step (int): Reduces the data on the time axis by factor t_step. Uses mean averaging. Default is 1. 
+    channel_step (int): Like t_step, but for the sensor position.
+Returns:
+    A 2d-numpy-array containing the data.
+    The first axis corresponds to the time, the second to the channel (sensor position)
+ ```
+
+For more details have a look at the inline documentation of [chunk.py](src/das2numpy/chunk.py)
+
+
+## Command Line Interface
+
+Creates a numpy file from the requested data. Optionally, the binary data can be printed to stdout.
+
+Example call:
+```
+python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2024-DESY/2024-07-23-desy 2024-07-23T10:01:00 2024-07-23T10:02:00 10 0 1000 10 default
+```
+
+For more information:
+```
+python -m das2numpy -h
+```
+
+
+## Issues
+
+- Loading from OPTASENSE may not work anymore. I haven't tested it for a long time.

{das2numpy-1.0 → das2numpy-1.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "das2numpy"
-version = "1.0"
+version = "1.1"
 authors = [
   { name="Erik Genthe", email="erik.genthe@desy.de" },
 ]
@@ -16,6 +16,8 @@ classifiers = [
     "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
     "Operating System :: OS Independent",
 ]
-
+dependencies = [
+    "numpy", "ffmpeg-python", "h5py", "scipy", "numba",
+]
 [project.urls]
 Homepage = "https://git.physnet.uni-hamburg.de/wave/das2numpy"

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy/__init__.py

@@ -8,13 +8,21 @@ from . import utils
 
 
 def loader(root_path:str, predefined_setup:str, num_worker_threads):
-
+    """ 
+    Loads data and returns it as a numpy array. 
+    Args:
+        root_path (str): Path to directory that contains the files to be loaded from. Subdirectories are (recursively) also searched.
+        predefined_setup (str): One of ["SILIXA", "FLAC_200HZ", "OPTASENSE"]
+        num_worker_threads (int): The number of worker threads used for loading files in parallel.
+    Returns:
+        A loader instance to load data. Call instance.load_array(...).
+    """
     if predefined_setup.upper() == "SILIXA":
         from .setups import silixa
         chunk = silixa.init(root_path, num_worker_threads)
-    elif predefined_setup.upper() == "SILIXA_200HZ":
-        from .setups import silixa_200hz
-        chunk = silixa_200hz.init(root_path, num_worker_threads)
+    #elif predefined_setup.upper() == "SILIXA_200HZ":
+    #    from .setups import silixa_200hz
+    #    chunk = silixa_200hz.init(root_path, num_worker_threads)
     elif predefined_setup.upper() == "FLAC_200HZ":
         from .setups import flac_200hz
         chunk = flac_200hz.init(root_path, num_worker_threads)

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy/__main__.py

@@ -13,8 +13,12 @@ def parse_arguments():
         default=False,
         help="Print more information to stdout"
     )
-
-
+    parser.add_argument(
+        "--workers",
+        type=int,
+        default=1,
+        help="Number of worker threads used for loading files in parallel."
+    )
     parser.add_argument(
         "device",
         type=str,
@@ -38,7 +42,7 @@ def parse_arguments():
     parser.add_argument(
         "time_step",
         type=int,
-        help="Time step as an integer."
+        help="Time step as an integer. Uses mean averaging."
     )
     parser.add_argument(
         "channel_start",
@@ -53,7 +57,7 @@ def parse_arguments():
     parser.add_argument(
         "channel_step",
         type=int,
-        help="Channel step as an integer."
+        help="Channel step as an integer. Uses mean averaging."
     )
     parser.add_argument(
         "output",
@@ -81,9 +85,8 @@ def main():
 
     print("Load...")
     start = time()
-    loaderinstance = loader(args.root_path, args.device, num_worker_threads=1)
-    data = loaderinstance.load_array(args.start, args.end, args.time_step,
-            args.channel_start, args.channel_end, args.channel_step)
+    loaderinstance = loader(args.root_path, args.device, num_worker_threads=args.workers)
+    data = loaderinstance.load_array(args.start, args.end, args.channel_start, args.channel_end, args.time_step, args.channel_step)
     if args.verbosity:
         end = time()
         print("Duration", end-start)

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy/chunk.py

@@ -13,7 +13,6 @@ from typing import Callable
 from math import floor
 from datetime import datetime
 from random import shuffle
-from multipledispatch import dispatch
 import concurrent.futures as CF
 from concurrent.futures import ThreadPoolExecutor
 from threading import Lock
@@ -59,8 +58,8 @@ class Chunk():
         assert type(sample_rate) == int
         if multithreaded:
             self.__executor = ThreadPoolExecutor(workers)
-        if not self.__multithreaded:
-            print("Warning: Chunk is not in multiprocessing or multithreading mode!")
+        #if not self.__multithreaded:
+        #    print("Warning: Chunk is not in multiprocessing or multithreading mode!")
 
 
 
@@ -114,7 +113,6 @@ class Chunk():
         n_channels = min(data.shape[1], self.data.shape[1])
         self.data[start_index : start_index + data.shape[0], 0:n_channels] = data[:,:n_channels]
     
-    @dispatch(int, int, int, int, int, int)
     def load_array_posix_ms(self, t_start: int, t_end: int, t_step: int, channel_start: int, channel_end: int, channel_step: int) -> NP.ndarray:
         """ Loading data
             Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
@@ -197,50 +195,21 @@ class Chunk():
 
 
 
-    @dispatch(datetime, datetime, int, int)
-    def load_array(self, t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
-        """ Loads data and returns it as a numpy array. 
-            Constraints: 
-                t_start has to be less or equal t_end, 
-                same for channel_start and channel_end.
-            Args:
-                t_start (datetime): datetime object which defines the start of the data to load.
-                t_end (datetime): datetime object which defines the end of the data to load.
-                channel_start (int): The starting index of sensor in the data (inclusive).
-                channel_end (int): The ending index of sensors in the data (exclusive).
-            Returns:
-                A 2d-numpy-array containing the data.
-                The first axis corresponds to the time, the second to the channel
-        """
-        return self.load_array(t_start, t_end, 1, channel_start, channel_end, 1)
-
-
-    @dispatch(datetime, datetime, int, int, int, int)
-    def load_array(self, t_start:datetime, t_end:datetime, t_step:int, channel_start:int, channel_end:int, channel_step:int) -> NP.ndarray:
+    def load_array(self, t_start:datetime, t_end:datetime, channel_start:int, channel_end:int, t_step=1, channel_step=1) -> NP.ndarray:
         """ Loading data into numpy array.
             Returns nothing, the data can be accessed by accessing the data field of this instance.
             Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
                     Consider using multithreaded=True in the constructor and a high amount of workers if needed.
-            Constraints: 
-                t_start has to be less or equal t_end, 
-                same for channel_start and channel_end.
-                t_step and channel_step have to be greater then 0
             Args:
                 t_start (datetime): datetime object which defines the start of the data to load.
                 t_end (datetime): datetime object which defines the end of the data to load.
-                t_step (int): If you, for example only want to load the data of every fourth timestep use t_end=4
-                channel_start (int): The starting index of sensor in the data (inclusive).
-                channel_end (int): The ending index of sensors in the data (exclusive).
+                channel_start (int): The starting index of the sensor position in the data (inclusive).
+                channel_end (int): The ending index of the sensors position in the data (exclusive).
+                t_step (int): Reduces the data on the time axis by factor t_step. Uses mean averaging. Default is 1. 
                 channel_step (int): Like t_step, but for the sensor position.
             Returns:
                 A 2d-numpy-array containing the data.
-                The first axis corresponds to the time, the second to the channel
+                The first axis corresponds to the time, the second to the channel (sensor position)
         """
         return self.load_array_posix_ms(to_posix_timestamp_ms(t_start), to_posix_timestamp_ms(t_end), t_step, channel_start, channel_end, channel_step)
 
-
-    @dispatch(int, int, int, int)
-    def load_array_posix_ms(self, t_start:int, t_end:int, channel_start:int, channel_end:int) -> NP.ndarray:
-        return self.load_array_posix_ms(t_start, t_end, 1, channel_start, channel_end, 1)
-
-

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy/setups/light_tdms_reader.py

@@ -24,19 +24,10 @@ Changed by Erik Genthe, erik.genthe@desy.de
 """
 
 import os, struct, datetime
-import pandas as pd
 import numpy as np
 import mmap
-
-import matplotlib.pyplot as plt
 from copy import deepcopy
 
-#%%
-def load_property_map(xls_file):
-    prop_map = pd.read_excel(xls_file, sheetname='Sheet1')
-    return prop_map[['CurrentTag', 'CorrectTag']].applymap(lambda x: x.replace(" ", "")).set_index('CurrentTag').to_dict()['CorrectTag']
-
-#prop_map = load_property_map('MetaDataTable_iDAS_TDMS_CFG_Tags.xlsx')
 
 def write_property_dict(prop_dict, out_file):
     from pprint import pformat
@@ -180,34 +171,19 @@ class TdmsReader(object):
 
     channel_length = property(_get_channel_length)
 
-    def get_properties(self, mapped=False):
+    def get_properties(self):
         """
         Return a dictionary of properties. Read from file only if necessary.
         """
         # Check if already hold properties in memory
         if self._properties is None:
             self._properties = self._read_properties()
-        if mapped:
-            props = self._properties.copy()
-            tmp = [prop_map.get(col.replace(" ", ""),col.replace(" ", "")) for col in self._properties.index]
-            tmp1 = []
-            def addToList(ls, val, cnt=0):
-                if val not in ls:
-                    ls.append(val)
-                else:
-                    newVal = val + '_' + str(cnt+1)
-                    if newVal not in ls:
-                        ls.append(newVal)
-                    else:
-                        addToList(ls, val, cnt+1)
-
-            for col in tmp:
-                addToList(tmp1, col)
-
-            props.index = tmp1
-            return props.loc[:,'Value'].to_dict()
-        else:
-            return self._properties.loc[:,'Value'].to_dict()
+        print(self._properties)
+        dict = {}
+        for key, _, value in self._properties:
+            dict[key] = value
+        return dict
+
 
     def _read_property(self):
         """ 
@@ -242,15 +218,12 @@ class TdmsReader(object):
 
         # loop through and read each property
         properties = [self._read_property() for _ in range(var)]
-        df = pd.DataFrame(properties)
-        df.columns = ['Property', 'Type', 'Value']
-        df.set_index('Property', inplace=True)
 
         self._end_of_properties_offset = self._tdms_file.tell()
 
         self._read_chunk_size()
         #TODO: Add number of channels to properties
-        return df
+        return properties
 
     def _read_chunk_size(self):
         """ Read the data chunk size from the TDMS file header."""

das2numpy-1.1/src/das2numpy.egg-info/PKG-INFO

@@ -0,0 +1,102 @@
+Metadata-Version: 2.4
+Name: das2numpy
+Version: 1.1
+Summary: A simple and universal package for loading large amounts of distributed acoustic sensing (DAS) data.
+Author-email: Erik Genthe <erik.genthe@desy.de>
+Project-URL: Homepage, https://git.physnet.uni-hamburg.de/wave/das2numpy
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: ffmpeg-python
+Requires-Dist: h5py
+Requires-Dist: scipy
+Requires-Dist: numba
+Dynamic: license-file
+
+# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
+
+
+
+## Install
+
+You can install via PIP.
+```
+python -m pip install das2numpy
+```
+
+To load data from flac files, ffmpeg (https://ffmpeg.org) needs to be installed. It is not possible to install ffmpeg with pip.
+
+On DESY's Maxwell cluster ffmpeg is available as a module. Before using das2numpy execute:
+```
+module load maxwell ffmpeg
+```
+
+
+
+
+## Python API
+
+Example: If you want to get started quickly, have a look at the [example.py](src/example.py).
+
+Create an instance with:
+
+```python
+def loader(root_path:str, predefined_setup:str, num_worker_threads):
+```
+```
+    Loads data and returns it as a numpy array. 
+    Args:
+        root_path (str): Path to directory that contains the files to be loaded from. Subdirectories are (recursively) also searched.
+        predefined_setup (str): One of ["SILIXA", "FLAC_200HZ", "OPTASENSE"]
+        num_worker_threads (int): The number of worker threads used for loading files in parallel.
+    Returns:
+        A loader instance to load data. Call instance.load_array(...).
+```
+
+Use one of the load_array(..) functions of that instance.
+
+```python
+def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
+```
+```
+Loading data into numpy array.
+Returns nothing, the data can be accessed by accessing the data field of this instance.
+Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
+        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
+Args:
+    t_start (datetime): datetime object which defines the start of the data to load.
+    t_end (datetime): datetime object which defines the end of the data to load.
+    channel_start (int): The starting index of the sensor position in the data (inclusive).
+    channel_end (int): The ending index of the sensors position in the data (exclusive).
+    t_step (int): Reduces the data on the time axis by factor t_step. Uses mean averaging. Default is 1. 
+    channel_step (int): Like t_step, but for the sensor position.
+Returns:
+    A 2d-numpy-array containing the data.
+    The first axis corresponds to the time, the second to the channel (sensor position)
+ ```
+
+For more details have a look at the inline documentation of [chunk.py](src/das2numpy/chunk.py)
+
+
+## Command Line Interface
+
+Creates a numpy file from the requested data. Optionally, the binary data can be printed to stdout.
+
+Example call:
+```
+python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2024-DESY/2024-07-23-desy 2024-07-23T10:01:00 2024-07-23T10:02:00 10 0 1000 10 default
+```
+
+For more information:
+```
+python -m das2numpy -h
+```
+
+
+## Issues
+
+- Loading from OPTASENSE may not work anymore. I haven't tested it for a long time.

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy.egg-info/SOURCES.txt

@@ -2,16 +2,15 @@ LICENSE
 README.md
 pyproject.toml
 src/example.py
-src/test_downsampled.py
 src/das2numpy/__init__.py
 src/das2numpy/__main__.py
 src/das2numpy/chunk.py
 src/das2numpy/filefinder.py
-src/das2numpy/test.py
 src/das2numpy/utils.py
 src/das2numpy.egg-info/PKG-INFO
 src/das2numpy.egg-info/SOURCES.txt
 src/das2numpy.egg-info/dependency_links.txt
+src/das2numpy.egg-info/requires.txt
 src/das2numpy.egg-info/top_level.txt
 src/das2numpy/setups/flac_200hz.py
 src/das2numpy/setups/light_tdms_reader.py

das2numpy-1.1/src/das2numpy.egg-info/requires.txt

@@ -0,0 +1,5 @@
+numpy
+ffmpeg-python
+h5py
+scipy
+numba

{das2numpy-1.0 → das2numpy-1.1}/src/das2numpy.egg-info/top_level.txt

@@ -1,3 +1,2 @@
 das2numpy
 example
-test_downsampled

{das2numpy-1.0 → das2numpy-1.1}/src/example.py

@@ -6,21 +6,24 @@ from das2numpy import loader, utils
 
 
 print("Load data to numpy-array")
-t_start = datetime(2025,  3, 25, 1, 0, 0)
-t_end   = datetime(2025,  3, 25, 1, 1, 0)
+t_start = datetime(2024, 7, 23, 1, 0, 0)
+t_end   = datetime(2024, 7, 23, 1, 1, 0)
 channel_start = 0
 channel_end = -1
-loader = loader("/pnfs/desy.de/m/project/iDAS/raw/2025-DESY/2025-03-25-desy", "SILIXA", 1)
+#loader = loader("/pnfs/desy.de/m/project/iDAS/raw/2024-DESY/2024-07-23-desy", "SILIXA", 1) # 1000 Hz
+loader = loader("/pnfs/desy.de/m/project/iDAS/work/IDAS_200HZ/", "FLAC_200HZ", 1) # 200 Hz
 data = loader.load_array(t_start, t_end, channel_start, channel_end)
 
 print("Reduce data by binning (mean averaging)")
 bin_factors = (100, 10)
 data = utils.bin(data, bin_factors) # Reduce time sampling and spatial sampling by averaging.
-sampling_hz = 1000.0 / bin_factors[0]
+sampling_hz = 200.0 / bin_factors[0]
 channel_spacing = 1.0 * bin_factors[1]
 
-NP.save("data.npy", data)
+# Saving loaded data to numpy file
+NP.save("data.npy", data) 
 
+# Creating a waterfall plot
 print("Create plot with pyplot")
 PP.title(f"{t_start.isoformat()}")
 PP.imshow(

das2numpy-1.0/PKG-INFO

@@ -1,93 +0,0 @@
-Metadata-Version: 2.4
-Name: das2numpy
-Version: 1.0
-Summary: A simple and universal package for loading large amounts of distributed acoustic sensing (DAS) data.
-Author-email: Erik Genthe <erik.genthe@desy.de>
-Project-URL: Homepage, https://git.physnet.uni-hamburg.de/wave/das2numpy
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Dynamic: license-file
-
-# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
-
-Python: If you want to get started quickly, have a look at the [example.py](src/example.py).
-
-
-## Install
-
-You can install via PIP.
-```
-python -m pip install das2numpy
-```
-
-If you want to run the source have a look at *install_dependencies.sh*.
-
-
-## Use as python module
-### API
-
-
-#### Recommended: simplest interface
-```python
-def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
-```
-
-```
-Loads data and returns it as a numpy array. 
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
- ```
-
-
-
-#### More detailed interface
-```python
-def load_array(t_start:datetime, t_end:datetime, t_step:int, channel_start:int, channel_end:int, channel_step:int) -> NP.ndarray:
-```
-
-``` Loading data into numpy array.
-Returns nothing, the data can be accessed by accessing the data field of this instance.
-Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
-        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
-Constraints: 
-    t_start has to be less or equal t_end, 
-    same for channel_start and channel_end.
-    t_step and channel_step have to be greater then 0
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    t_step (int): If you, for example only want to load the data of every fourth timestep use t_end=4
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-    channel_step (int): Like t_step, but for the sensor position.
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
-```
-
-### Lower level interfaces
-There are also lower level interfaces in the module.
-For example, the above interfaces also exist with POSIX timestamps in milliseconds instead of datetime objects. These timestamps have exactly the same resolution as the time axis of the resulting array.
-
-
-## Use as command line interface
-
-Example call:
-```
-python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2025-DESY/2025-03-25-desy 2025-03-25T10:01:00 2025-03-25T10:02:00 10 0 1000 10 default
-```
-
-For more information:
-```
-python -m das2numpy -h
-```

das2numpy-1.0/README.md

@@ -1,79 +0,0 @@
-# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
-
-Python: If you want to get started quickly, have a look at the [example.py](src/example.py).
-
-
-## Install
-
-You can install via PIP.
-```
-python -m pip install das2numpy
-```
-
-If you want to run the source have a look at *install_dependencies.sh*.
-
-
-## Use as python module
-### API
-
-
-#### Recommended: simplest interface
-```python
-def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
-```
-
-```
-Loads data and returns it as a numpy array. 
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
- ```
-
-
-
-#### More detailed interface
-```python
-def load_array(t_start:datetime, t_end:datetime, t_step:int, channel_start:int, channel_end:int, channel_step:int) -> NP.ndarray:
-```
-
-``` Loading data into numpy array.
-Returns nothing, the data can be accessed by accessing the data field of this instance.
-Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
-        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
-Constraints: 
-    t_start has to be less or equal t_end, 
-    same for channel_start and channel_end.
-    t_step and channel_step have to be greater then 0
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    t_step (int): If you, for example only want to load the data of every fourth timestep use t_end=4
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-    channel_step (int): Like t_step, but for the sensor position.
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
-```
-
-### Lower level interfaces
-There are also lower level interfaces in the module.
-For example, the above interfaces also exist with POSIX timestamps in milliseconds instead of datetime objects. These timestamps have exactly the same resolution as the time axis of the resulting array.
-
-
-## Use as command line interface
-
-Example call:
-```
-python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2025-DESY/2025-03-25-desy 2025-03-25T10:01:00 2025-03-25T10:02:00 10 0 1000 10 default
-```
-
-For more information:
-```
-python -m das2numpy -h
-```

das2numpy-1.0/src/das2numpy/test.py

@@ -1,158 +0,0 @@
-"""
-    Deprecated
-    
-    Unittests for this dataloader-module
-    by Erik Genthe
-    05.01.2022
-"""
-from math import ceil, floor
-import sys as SYS
-from os import path as P
-import datetime as DT
-import h5py as H5PY
-import numpy as NP
-
-try:
-    import dataloader as D
-except ModuleNotFoundError as e:
-    raise RuntimeError("TO RUN THIS TEST, MOVE IT INTO THE PARENT DIR FIRST!") from e
-from dataloader.filefinder import to_posix_timestamp_ms
-
-
-
-def test_silixa_filefinder():
-    #file_path = '/wave/seismic-rawdata/desy_12km_1m_P7gauss/desy_UTC_20210522_155121.950.tdms'
-    #ls /wave/seismic-rawdata/desy_12km_1m_P7gauss -l | grep -n --invert-match 504946688
-
-    # Find one specific file...
-    time = DT.datetime(2021, 5, 30, 14, 00, 00)
-    filelist = D.silixa.FILE_FINDER.get_range(time, time)
-    assert len(filelist) == 1
-    assert filelist[0][1].endswith('/desy_UTC_20210530_135950.619.tdms')
-
-    # Find all files...
-    filelist = D.silixa.FILE_FINDER.get_range_posix(0, D.to_posix_timestamp_ms(DT.datetime.now()))
-    assert len(filelist) > 9000
-
-
-def test_optasense_filefinder():
-    # Find one specific file...
-    time = DT.datetime(2021, 5, 30, 14, 00, 00)
-    filelist = D.optasense.FILE_FINDER.get_range(time, time)
-    assert len(filelist) == 1
-    assert filelist[0][1].endswith('2021-05-30T135924Z.h5')
-
-    # Find all files...
-    filelist = D.optasense.FILE_FINDER.get_range_posix(0, D.to_posix_timestamp_ms(DT.datetime.now()))
-    assert len(filelist) > 9000
-
-
-def test_fast_optasense_filefinder():
-    # Find one specific file...
-    time = DT.datetime(2021, 5, 30, 14, 00, 00)
-    filelist = D.fast_optasense.FILE_FINDER.get_range(time, time)
-    assert len(filelist) == 1
-    assert filelist[0][1].endswith('2021-05-30T135924Z.h5.bin')
-
-    # Find all files...
-    filelist = D.optasense.FILE_FINDER.get_range_posix(0, D.to_posix_timestamp_ms(DT.datetime.now()))
-    assert len(filelist) > 9000
-
-
-
-def test_chunk(chunk, MAX_CHANNEL):
-    import time as TIME
-    #MAX_CHANNEL = 12608
-    #chunk = D.silixa.create_chunk()
-    t_start: int =          to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00, 00))
-    t_end1: int = to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00,  1))
-    t_end2: int = to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14,  1, 30))
-    t_end3: int = to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 10, 00))
-    t_end_one_hour: int =   to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 15, 00, 00))
-    print()
-
-    chunk.load(t_start, t_end1, 1, 0, MAX_CHANNEL, 1)
-    assert chunk.data.shape == (1000, MAX_CHANNEL)
-    print()
-
-    chunk.load(t_start, t_end2, 3, 0, MAX_CHANNEL, 9)
-    assert chunk.data.shape == (30000, ceil(MAX_CHANNEL / 9))
-    print()
-
-    # Now some benchmarks...
-    #bench_start = TIME.time()
-    #file_handle = open("/wave/seismic-rawdata/OPTA/Disk2/DESY-Rec-11-GL8m-Chan10000_2021-05-30T07_55_42+0100/DESY-Rec-11-GL8m-Chan10000_2021-05-30T135924Z.h5", 'rb')
-    #file:H5PY.File = H5PY.File(file_handle, 'r')
-    #data = file['Acquisition']['Raw[0]']['RawData'] # Data is not loaded into memory at this point! (Lazy evaluation)
-    #data = NP.array(data)
-    #print("TIME for loading one whole file using h5py:", TIME.time() - bench_start, "\n")
-
-    bench_start = TIME.time()
-    chunk.load(t_start, t_end3, 1, 0, 1000, 1)
-    print("Time for loading the first 1000 sensors of one hour of data: %4f\n" % (TIME.time() - bench_start))
-    assert chunk.data.shape == (600000, 1000)
-
-    bench_start = TIME.time()
-    chunk.load(t_start, t_end_one_hour, 1, 0, MAX_CHANNEL, 10)
-    print("Time for loading one hour of data with with sensor_step=10: %4f\n" % (TIME.time() - bench_start))
-    assert chunk.data.shape == (1000*60*60, ceil(MAX_CHANNEL/10))
-
-    bench_start = TIME.time()
-    chunk.load(t_start, t_end_one_hour, 1, 0, 100, 1)
-    print("Time for loading 100 sensors with 1 hour of data: %4f\n" % (TIME.time() - bench_start))
-
-    bench_start = TIME.time()
-    chunk.load(t_start, t_end_one_hour, 1, 0, 1000, 1)
-    print("Time for loading 1000 sensors with 1 hour of data: %4f\n" % (TIME.time() - bench_start))
-
-    bench_start = TIME.time()
-    chunk.load(t_start, t_end_one_hour, 1, 0, MAX_CHANNEL, 1)
-    print("Time for loading 1 hour completely: %4f\n" % (TIME.time() - bench_start))
-
-
-
-def test_equalness_of_fast_opta_simple():
-    t_start: int =  to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00, 00))
-    t_end: int =    to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00,  1))
-
-    chunk_fast = D.fast_optasense.create_chunk()
-    chunk_fast.load(t_start, t_end, 1, 0, 10, 1)
-
-    chunk_normal = D.optasense.create_chunk()
-    chunk_normal.load(t_start, t_end, 1, 0, 10, 1)
-
-    assert chunk_fast.data.shape == chunk_normal.data.shape
-    assert NP.array_equiv(chunk_fast.data, chunk_normal.data)
-
-
-def test_equalness_of_fast_opta():
-    t_start: int =  to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00, 00))
-    t_end: int =    to_posix_timestamp_ms(DT.datetime(2021, 5, 30, 14, 00,  1))
-
-    chunk_fast = D.fast_optasense.create_chunk()
-    chunk_fast.load(t_start, t_end, 3, 2000, 7000, 9)
-
-    chunk_normal = D.optasense.create_chunk()
-    chunk_normal.load(t_start, t_end, 3, 2000, 7000, 9)
-
-    assert chunk_fast.data.shape == chunk_normal.data.shape
-    assert NP.array_equiv(chunk_fast.data, chunk_normal.data)
-
-
-if __name__ == '__main__':
-    #test_equalness_of_fast_opta_simple()
-    #test_equalness_of_fast_opta()
-    #test_fast_optasense_filefinder()
-    #test_silixa_filefinder()
-    #test_optasense_filefinder()
-
-
-
-    print("\nSilixa benchmark:")
-    test_chunk(D.silixa.create_chunk(), 12608)
-
-    print("\nFast Optasense benchmark:")
-    test_chunk(D.fast_optasense.create_chunk(), 10000)
-
-    #print("\nOptasense benchmark:")
-    #test_chunk(D.optasense.create_chunk(), 10000)

das2numpy-1.0/src/das2numpy.egg-info/PKG-INFO

@@ -1,93 +0,0 @@
-Metadata-Version: 2.4
-Name: das2numpy
-Version: 1.0
-Summary: A simple and universal package for loading large amounts of distributed acoustic sensing (DAS) data.
-Author-email: Erik Genthe <erik.genthe@desy.de>
-Project-URL: Homepage, https://git.physnet.uni-hamburg.de/wave/das2numpy
-Classifier: Programming Language :: Python :: 3
-Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Dynamic: license-file
-
-# Module for loading Distributed Acoustic Sensing (DAS) data. SILIXA / OPTASENSE
-
-Python: If you want to get started quickly, have a look at the [example.py](src/example.py).
-
-
-## Install
-
-You can install via PIP.
-```
-python -m pip install das2numpy
-```
-
-If you want to run the source have a look at *install_dependencies.sh*.
-
-
-## Use as python module
-### API
-
-
-#### Recommended: simplest interface
-```python
-def load_array(t_start:datetime, t_end:datetime, channel_start:int, channel_end:int) -> NP.ndarray:
-```
-
-```
-Loads data and returns it as a numpy array. 
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
- ```
-
-
-
-#### More detailed interface
-```python
-def load_array(t_start:datetime, t_end:datetime, t_step:int, channel_start:int, channel_end:int, channel_step:int) -> NP.ndarray:
-```
-
-``` Loading data into numpy array.
-Returns nothing, the data can be accessed by accessing the data field of this instance.
-Warning: using a different value then 1 for t_step or channel_step can result in a high cpu-usage.
-        Consider using multithreaded=True in the constructor and a high amount of workers if needed.
-Constraints: 
-    t_start has to be less or equal t_end, 
-    same for channel_start and channel_end.
-    t_step and channel_step have to be greater then 0
-Args:
-    t_start (datetime): datetime object which defines the start of the data to load.
-    t_end (datetime): datetime object which defines the end of the data to load.
-    t_step (int): If you, for example only want to load the data of every fourth timestep use t_end=4
-    channel_start (int): The starting index of sensor in the data (inclusive).
-    channel_end (int): The ending index of sensors in the data (exclusive).
-    channel_step (int): Like t_step, but for the sensor position.
-Returns:
-    A 2d-numpy-array containing the data.
-    The first axis corresponds to the time, the second, to the channel
-```
-
-### Lower level interfaces
-There are also lower level interfaces in the module.
-For example, the above interfaces also exist with POSIX timestamps in milliseconds instead of datetime objects. These timestamps have exactly the same resolution as the time axis of the resulting array.
-
-
-## Use as command line interface
-
-Example call:
-```
-python -m das2numpy "SILIXA" /pnfs/desy.de/m/project/iDAS/raw/2025-DESY/2025-03-25-desy 2025-03-25T10:01:00 2025-03-25T10:02:00 10 0 1000 10 default
-```
-
-For more information:
-```
-python -m das2numpy -h
-```

das2numpy-1.0/src/test_downsampled.py

@@ -1,54 +0,0 @@
-import numpy as NP
-import sys
-from datetime import datetime
-import matplotlib.pyplot as PP
-from das2numpy import loader, utils
-
-USE_DOWNSAMPLED = False
-
-print("Load data to numpy-array")
-t_start = datetime(2025, 10, 14, 2, 58, 59)
-t_end   = datetime(2025, 10, 14, 2, 59, 1)
-channel_start = 1000
-channel_end = 3000
-
-if USE_DOWNSAMPLED:
-    loader = loader("/pnfs/desy.de/m/project/iDAS/work/derived-data/DOWNSAMPLED_200HZ/2025-10/", "SILIXA_200HZ", 1)
-else:
-    loader = loader("/pnfs/desy.de/m/project/iDAS/raw/2025-DESY/2025-10-14-desy", "SILIXA", 1)
-data = loader.load_array(t_start, t_end, channel_start, channel_end)
-
-print("Reduce data by binning (mean averaging)")
-if USE_DOWNSAMPLED:
-    bin_factors = (1, 1)
-    data = utils.bin(data, bin_factors) # Reduce time sampling and spatial sampling by averaging.
-    sampling_hz = 200.0 / bin_factors[0]
-else:
-    bin_factors = (5, 1)
-    data = utils.bin(data, bin_factors) # Reduce time sampling and spatial sampling by averaging.
-    sampling_hz = 1000.0 / bin_factors[0]
-channel_spacing = 1.0 * bin_factors[1]
-
-NP.save("data.npy", data)
-
-print("Create plot with pyplot")
-PP.title(f"{t_start.isoformat()}")
-PP.imshow(
-    data,
-    cmap = "seismic",
-    aspect = "auto",
-    interpolation = "nearest",
-    vmin = -1e-7,
-    vmax = +1e-7,
-    extent = (
-        channel_start, channel_start + (data.shape[1] * channel_spacing), 
-        data.shape[0] / sampling_hz, 0
-    )
-)
-PP.xlabel("Position [m]")
-PP.ylabel("Time [s]")
-PP.colorbar(label="Strain-rate [$\\frac{m}{m \\cdot s}$]")
-if USE_DOWNSAMPLED:
-    PP.savefig("waterfall_downsampled.png")
-else:
-    PP.savefig("waterfall.png")