daplis 0.9.1__tar.gz

daplis-0.9.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Sergei Kulkov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

daplis-0.9.1/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.1
+Name: daplis
+Version: 0.9.1
+Summary: Data Analysis Package for LInoSpad2
+Author-email: Sergei Kulkov <sergei.kulkov23@gmail.com>
+License: MIT
+Keywords: timestamp,analysis,LinoSPAD2
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: lmfit>=0.9.1
+Requires-Dist: matplotlib>=3.5.0
+Requires-Dist: numpy>=1.22.4
+Requires-Dist: pandas>=1.4.0
+Requires-Dist: pyarrow>=17.0.0
+Requires-Dist: scipy>=1.7.0
+Requires-Dist: seaborn>=0.9.0
+Requires-Dist: tqdm>=4.5.0
+
+# LinoSPAD2
+
+Package for unpacking and analyzing the binary data from LinoSPAD2.
+
+![Tests](https://github.com/rngKomorebi/LinoSPAD2/actions/workflows/tests.yml/badge.svg)
+![Documentation](https://github.com/rngKomorebi/LinoSPAD2/actions/workflows/documentation.yml/badge.svg)
+
+## Introduction
+
+This package was written for data analysis for LinoSPAD2, mainly for
+analysis of the timestamp output. The key functions are ones for
+unpacking the binary output of the detector that utilizes the numpy
+Python library for quick unpacking of .dat files to matrices,
+dictionaries, or data frames.
+
+The "functions" folder holds all functions from unpacking to plotting
+numerous types of graphs (pixel population, histograms of timestamp
+differences, etc.)
+
+The "params" folder holds masks (used to mask some of the noisiest
+pixels) and calibration data (compensating for TDC nonlinearities and
+offset) for LinoSPAD2 daughterboards "A5" and "NL11".
+
+The "archive" folder is a collection of scripts for debugging, tests,
+older versions of functions, etc.
+
+Full documentation, including examples and full documentation of
+modules and functions, can be found [here](https://rngkomorebi.github.io/LinoSPAD2/).
+
+Some functions (mainly the plotting ones) save plots as pictures in the
+.png format, creating a folder for the output in the same folder that
+holds the data. Others (such as delta_t.py for collecting timestamp differences
+in the given time window) save .csv files with the processed data for
+easier and faster plotting.
+
+Additionally, a standalone repo with an application for online plotting
+of the sensor population can be found [here](https://github.com/rngKomorebi/LinoSPAD2-app).
+
+## Installation and usage
+
+To start using the package, one can download the whole repo. The 'main.py'
+serves as the main hub for calling the functions. "requirements.txt"
+collects all packages required for this project to run. One can create
+an environment for this project either using conda or install the
+necessary packages using pip (for creating virtual environments using pip
+see [this](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)):
+```
+pip install virtualenv
+py -m venv PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME
+PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME/Scripts/activate
+cd PATH/TO/GITHUB/CODES/LinoSPAD2
+pip install -r requirements.txt
+```
+or (recommended)
+```
+conda create --name NEW_ENVIRONMENT_NAME --file /PATH/TO/requirements.txt -c conda-forge
+```
+To install the package, first, switch to the created environment:
+```
+conda activate NEW_ENVIRONMENT_NAME
+```
+or
+```
+PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME/Scripts/activate
+```
+and run
+```
+pip install -e .
+```
+that will install the local package LinoSPAD2. After that, you can
+import all functions in your project:
+```
+from LinoSPAD2.functions import plot_tmsp, delta_t, fits
+```
+
+## How to contribute
+
+This repo consists of two branches: 'main' serves as the release version
+of the package, tested, proven to be functional, and ready to use, while
+the 'develop' branch serves as the main hub for testing new stuff. To
+contribute, the best way would be to fork the 'develop' branch and
+submit via pull requests. Everyone willing to contribute is kindly asked
+to follow the [PEP 8](https://peps.python.org/pep-0008/) and
+[PEP 257](https://peps.python.org/pep-0257/) conventions.
+
+## License and contact info
+
+This package is available under the MIT license. See LICENSE for more
+information. If you'd like to contact me, the author, feel free to
+write at sergei.kulkov23@gmail.com.

daplis-0.9.1/README.md

@@ -0,0 +1,90 @@
+# LinoSPAD2
+
+Package for unpacking and analyzing the binary data from LinoSPAD2.
+
+![Tests](https://github.com/rngKomorebi/LinoSPAD2/actions/workflows/tests.yml/badge.svg)
+![Documentation](https://github.com/rngKomorebi/LinoSPAD2/actions/workflows/documentation.yml/badge.svg)
+
+## Introduction
+
+This package was written for data analysis for LinoSPAD2, mainly for
+analysis of the timestamp output. The key functions are ones for
+unpacking the binary output of the detector that utilizes the numpy
+Python library for quick unpacking of .dat files to matrices,
+dictionaries, or data frames.
+
+The "functions" folder holds all functions from unpacking to plotting
+numerous types of graphs (pixel population, histograms of timestamp
+differences, etc.)
+
+The "params" folder holds masks (used to mask some of the noisiest
+pixels) and calibration data (compensating for TDC nonlinearities and
+offset) for LinoSPAD2 daughterboards "A5" and "NL11".
+
+The "archive" folder is a collection of scripts for debugging, tests,
+older versions of functions, etc.
+
+Full documentation, including examples and full documentation of
+modules and functions, can be found [here](https://rngkomorebi.github.io/LinoSPAD2/).
+
+Some functions (mainly the plotting ones) save plots as pictures in the
+.png format, creating a folder for the output in the same folder that
+holds the data. Others (such as delta_t.py for collecting timestamp differences
+in the given time window) save .csv files with the processed data for
+easier and faster plotting.
+
+Additionally, a standalone repo with an application for online plotting
+of the sensor population can be found [here](https://github.com/rngKomorebi/LinoSPAD2-app).
+
+## Installation and usage
+
+To start using the package, one can download the whole repo. The 'main.py'
+serves as the main hub for calling the functions. "requirements.txt"
+collects all packages required for this project to run. One can create
+an environment for this project either using conda or install the
+necessary packages using pip (for creating virtual environments using pip
+see [this](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)):
+```
+pip install virtualenv
+py -m venv PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME
+PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME/Scripts/activate
+cd PATH/TO/GITHUB/CODES/LinoSPAD2
+pip install -r requirements.txt
+```
+or (recommended)
+```
+conda create --name NEW_ENVIRONMENT_NAME --file /PATH/TO/requirements.txt -c conda-forge
+```
+To install the package, first, switch to the created environment:
+```
+conda activate NEW_ENVIRONMENT_NAME
+```
+or
+```
+PATH/TO/ENVIRONMENT/ENVIRONMENT_NAME/Scripts/activate
+```
+and run
+```
+pip install -e .
+```
+that will install the local package LinoSPAD2. After that, you can
+import all functions in your project:
+```
+from LinoSPAD2.functions import plot_tmsp, delta_t, fits
+```
+
+## How to contribute
+
+This repo consists of two branches: 'main' serves as the release version
+of the package, tested, proven to be functional, and ready to use, while
+the 'develop' branch serves as the main hub for testing new stuff. To
+contribute, the best way would be to fork the 'develop' branch and
+submit via pull requests. Everyone willing to contribute is kindly asked
+to follow the [PEP 8](https://peps.python.org/pep-0008/) and
+[PEP 257](https://peps.python.org/pep-0257/) conventions.
+
+## License and contact info
+
+This package is available under the MIT license. See LICENSE for more
+information. If you'd like to contact me, the author, feel free to
+write at sergei.kulkov23@gmail.com.

daplis-0.9.1/pyproject.toml

@@ -0,0 +1,35 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "daplis"
+version = "0.9.1"
+description = "Data Analysis Package for LInoSpad2"
+authors = [{ name = "Sergei Kulkov", email = "sergei.kulkov23@gmail.com" }]
+license = { text = "MIT" }
+readme = "README.md"
+requires-python = ">=3.8"
+classifiers = [
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+]
+keywords = ["timestamp", "analysis", "LinoSPAD2"]
+
+dependencies = [
+    "lmfit>=0.9.1",
+    "matplotlib>=3.5.0",
+    "numpy>=1.22.4",
+    "pandas>=1.4.0",
+    "pyarrow>=17.0.0",
+    "scipy>=1.7.0",
+    "seaborn>=0.9.0",
+    "tqdm>=4.5.0",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]

daplis-0.9.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

daplis-0.9.1/src/LinoSPAD2/functions/calc_diff.py

@@ -0,0 +1,256 @@
+"""Module for computing timestamp differences.
+
+Compares all timestamps from the same cycle for the given pair of pixels
+against a given value (delta_window). Differences in that window are saved
+and returned as a list.
+
+This file can also be imported as a module and contains the following
+functions:
+
+    TODO remove
+    * calculate_differences_2212 - calculate timestamp differences for
+    the given pair of pixels. Works only with firmware version '2212'.
+    
+    * calculate_differences_2212_fast - calculate timestamp differences for
+    the given pair of pixels. Works only with firmware version '2212'.
+    Uses a faster algorithm than the function above.
+
+"""
+
+from typing import List
+from warnings import warn
+
+import numpy as np
+import pandas as pd
+from numpy import ndarray
+
+from LinoSPAD2.functions import utils
+
+
+def calculate_differences_2212(
+    data: List[float],
+    pixels: List[int] | List[List[int]],
+    pix_coor,
+    delta_window: float = 50e3,
+):
+    """Calculate timestamp differences for firmware version 2212.
+
+    Calculate timestamp differences for the given pixels and LinoSPAD2
+    firmware version 2212.
+
+    Parameters
+    ----------
+    data : list of array-like
+        List of data arrays, each corresponding to a different TDC.
+    pixels : List[int] | List[List[int]]
+        List of pixel numbers for which the timestamp differences should
+        be calculated or list of two lists with pixel numbers for peak
+        vs. peak calculations.
+    pix_coor : array-like
+        Array for transforming the pixel address in terms of TDC (0 to 3)
+        to pixel number in terms of half of the sensor (0 to 255).
+    delta_window : float, optional
+        Width of the time window for counting timestamp differences.
+        The default is 50e3 (50 ns).
+
+    Returns
+    -------
+    deltas_all : dict
+        Dictionary containing timestamp differences for each pair of pixels.
+
+    """
+
+    # TODO: remove
+    warn(
+        "This function is deprecated. Use" "'calculate_differences_2212_fast'",
+        DeprecationWarning,
+        stacklevel=2,
+    )
+    # Dictionary for the timestamp differences, where keys are the
+    # pixel numbers of the requested pairs
+    deltas_all = {}
+
+    pixels_left, pixels_right = utils.pixel_list_transform(pixels)
+
+    # Find ends of cycles
+    cycle_ends = np.argwhere(data[0].T[0] == -2)
+    cycle_ends = np.insert(cycle_ends, 0, 0)
+
+    for q in pixels_left:
+        # First pixel in the pair
+        tdc1, pix_c1 = np.argwhere(pix_coor == q)[0]
+        pix1 = np.where(data[tdc1].T[0] == pix_c1)[0]
+        for w in pixels_right:
+            if w <= q:
+                continue
+            deltas_all[f"{q},{w}"] = []
+
+            # Second pixel in the pair
+            tdc2, pix_c2 = np.argwhere(pix_coor == w)[0]
+            pix2 = np.where(data[tdc2].T[0] == pix_c2)[0]
+
+            # Go over cycles, getting data for the appropriate cycle
+            # only
+            for cyc in range(len(cycle_ends) - 1):
+                slice_from = cycle_ends[cyc]
+                slice_to = cycle_ends[cyc + 1]
+                pix1_slice = pix1[(pix1 >= slice_from) & (pix1 < slice_to)]
+                if not np.any(pix1_slice):
+                    continue
+                pix2_slice = pix2[(pix2 >= slice_from) & (pix2 < slice_to)]
+                if not np.any(pix2_slice):
+                    continue
+
+                # Calculate delta t
+                tmsp1 = data[tdc1].T[1][pix1_slice]
+                tmsp1 = tmsp1[tmsp1 > 0]
+                tmsp2 = data[tdc2].T[1][pix2_slice]
+                tmsp2 = tmsp2[tmsp2 > 0]
+                for t1 in tmsp1:
+                    deltas = tmsp2 - t1
+                    ind = np.where(np.abs(deltas) < delta_window)[0]
+                    deltas_all[f"{q},{w}"].extend(deltas[ind])
+
+    return deltas_all
+
+
+def calculate_differences_2212_fast(
+    data: ndarray,
+    pixels: List[int] | List[List[int]],
+    pix_coor: ndarray,
+    delta_window: float = 50e3,
+    cycle_length: float = 4e9,
+):
+    """Calculate timestamp differences for firmware version 2212.
+
+    Calculate timestamp differences for the given pixels and LinoSPAD2
+    firmware version 2212.
+
+    Parameters
+    ----------
+    data : ndarray
+        Matrix of timestamps, where rows correspond to the TDCs.
+    pixels : List[int] | List[List[int]]
+        List of pixel numbers for which the timestamp differences should
+        be calculated or list of two lists with pixel numbers for peak
+        vs. peak calculations.
+    pix_coor : ndarray
+        Array for transforming the pixel address in terms of TDC (0 to 3)
+        to pixel number in terms of half of the sensor (0 to 255).
+    delta_window : float, optional
+        Width of the time window for counting timestamp differences.
+        The default is 50e3 (50 ns).
+    cycle_length : float, optional
+        Length of each acquisition cycle. The default is 4e9 (4 ms).
+
+    Returns
+    -------
+    deltas_all : dict
+        Dictionary containing timestamp differences for each pair of pixels.
+
+    """
+
+    # Dictionary for the timestamp differences, where keys are the
+    # pixel numbers of the requested pairs
+    deltas_all = {}
+
+    pixels_left, pixels_right = utils.pixel_list_transform(pixels)
+
+    # Find ends of cycles
+    cycle_ends = np.argwhere(data[0].T[0] == -2)
+    cycle_ends = np.insert(cycle_ends, 0, 0)
+
+    for q in pixels_left:
+        # First pixel in the pair
+        tdc1, pix_c1 = np.argwhere(pix_coor == q)[0]
+        pix1 = np.where(data[tdc1].T[0] == pix_c1)[0]
+        for w in pixels_right:
+            if w <= q:
+                continue
+            deltas_all[f"{q},{w}"] = []
+
+            timestamps_1 = []
+            timestamps_2 = []
+
+            # Second pixel in the pair
+            tdc2, pix_c2 = np.argwhere(pix_coor == w)[0]
+            pix2 = np.where(data[tdc2].T[0] == pix_c2)[0]
+
+            # Go over cycles, shifting the timestamps from each next
+            # cycle by lengths of cycles before (e.g., for the 4th cycle
+            # add 12 ms)
+            for i, _ in enumerate(cycle_ends[:-1]):
+                slice_from = cycle_ends[i]
+                slice_to = cycle_ends[i + 1]
+                pix1_slice = pix1[(pix1 >= slice_from) & (pix1 < slice_to)]
+                if not np.any(pix1_slice):
+                    continue
+                pix2_slice = pix2[(pix2 >= slice_from) & (pix2 < slice_to)]
+                if not np.any(pix2_slice):
+                    continue
+
+                # Shift timestamps by cycle length
+                tmsp1 = data[tdc1].T[1][pix1_slice]
+                tmsp1 = tmsp1[tmsp1 > 0]
+                tmsp1 = tmsp1 + cycle_length * i
+
+                tmsp2 = data[tdc2].T[1][pix2_slice]
+                tmsp2 = tmsp2[tmsp2 > 0]
+                tmsp2 = tmsp2 + cycle_length * i
+
+                timestamps_1.extend(tmsp1)
+                timestamps_2.extend(tmsp2)
+
+            timestamps_1 = np.array(timestamps_1)
+            timestamps_2 = np.array(timestamps_2)
+
+            # Indicators for each pixel: 0 for timestamps from one pixel
+            # 1 - from the other
+            pix1_ind = np.zeros(len(timestamps_1), dtype=np.int32)
+            pix2_ind = np.ones(len(timestamps_2), dtype=np.int32)
+
+            pix1_data = np.vstack((pix1_ind, timestamps_1))
+            pix2_data = np.vstack((pix2_ind, timestamps_2))
+
+            # Dataframe for each pixel with pixel indicator and
+            # timestamps
+            df1 = pd.DataFrame(
+                pix1_data.T, columns=["Pixel_index", "Timestamp"]
+            )
+            df2 = pd.DataFrame(
+                pix2_data.T, columns=["Pixel_index", "Timestamp"]
+            )
+
+            # Combine the two dataframes
+            df_combined = pd.concat((df1, df2), ignore_index=True)
+
+            # Sort the timestamps
+            df_combined.sort_values("Timestamp", inplace=True)
+
+            # Subtract pixel indicators of neighbors; values of 0
+            # correspond to timestamp differences for the same pixel
+            # '-1' and '1' - to differences from different pixels
+            df_combined["Pixel_index_diff"] = df_combined["Pixel_index"].diff()
+
+            # Calculate timestamp difference between neighbors
+            df_combined["Timestamp_diff"] = df_combined["Timestamp"].diff()
+
+            # Get the correct timestamp difference sign
+            df_combined["Timestamp_diff"] = (
+                df_combined["Timestamp_diff"] * df_combined["Pixel_index_diff"]
+            )
+
+            # Collect timestamp differences where timestamps are from
+            # different pixels
+            filtered_df = df_combined[
+                abs(df_combined["Pixel_index_diff"]) == 1
+            ]
+
+            # Save only timestamps differences in the requested window
+            delta_ts = filtered_df[
+                abs(filtered_df["Timestamp_diff"]) < delta_window
+            ]["Timestamp_diff"].values
+
+            deltas_all[f"{q},{w}"].extend(delta_ts)
+
+    return deltas_all