gensor 0.0.1__tar.gz

gensor-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024, Mateusz Zawadzki
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gensor-0.0.1/PKG-INFO

@@ -0,0 +1,75 @@
+Metadata-Version: 2.1
+Name: gensor
+Version: 0.0.1
+Summary: Library for handling groundwater sensor data.
+Home-page: https://github.com/zawadzkim/gensor
+Author: Mateusz Zawadzki
+Author-email: fzawadzkimat@outlook.com
+Requires-Python: >=3.11
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: chardet (>=5.2.0,<6.0.0)
+Requires-Dist: matplotlib (>=3.9.2,<4.0.0)
+Requires-Dist: numpy (>=2.1.0,<3.0.0)
+Requires-Dist: pandas (>=2.2.2,<3.0.0)
+Requires-Dist: pandera (>=0.20.3,<0.21.0)
+Requires-Dist: pydantic (>=2.8.2,<3.0.0)
+Requires-Dist: pytz (>=2024.1,<2025.0)
+Requires-Dist: scikit-learn (>=1.5.1,<2.0.0)
+Requires-Dist: scipy (>=1.14.1,<2.0.0)
+Requires-Dist: sqlalchemy (>=2.0.32,<3.0.0)
+Project-URL: Documentation, https://zawadzkim.github.io/gensor/
+Project-URL: Repository, https://github.com/zawadzkim/gensor
+Description-Content-Type: text/markdown
+
+# gensor
+
+[![Release](https://img.shields.io/github/v/release/zawadzkim/gensor)](https://img.shields.io/github/v/release/zawadzkim/gensor)
+[![Build status](https://img.shields.io/github/actions/workflow/status/zawadzkim/gensor/main.yml?branch=main)](https://github.com/zawadzkim/gensor/actions/workflows/main.yml?query=branch%3Amain)
+[![codecov](https://codecov.io/gh/zawadzkim/gensor/branch/main/graph/badge.svg)](https://codecov.io/gh/zawadzkim/gensor)
+[![Commit activity](https://img.shields.io/github/commit-activity/m/zawadzkim/gensor)](https://img.shields.io/github/commit-activity/m/zawadzkim/gensor)
+[![License](https://img.shields.io/github/license/zawadzkim/gensor)](https://img.shields.io/github/license/zawadzkim/gensor)
+
+Library for handling groundwater sensor data.
+
+- **Github repository**: <https://github.com/zawadzkim/gensor/>
+- **Documentation** <https://zawadzkim.github.io/gensor/>
+
+## Getting started with your project
+
+First, create a repository on GitHub with the same name as this project, and then run the following commands:
+
+```bash
+git init -b main
+git add .
+git commit -m "init commit"
+git remote add origin git@github.com:zawadzkim/gensor.git
+git push -u origin main
+```
+
+Finally, install the environment and the pre-commit hooks with
+
+```bash
+make install
+```
+
+You are now ready to start development on your project!
+The CI/CD pipeline will be triggered when you open a pull request, merge to main, or when you create a new release.
+
+To finalize the set-up for publishing to PyPi or Artifactory, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/publishing/#set-up-for-pypi).
+For activating the automatic documentation with MkDocs, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/mkdocs/#enabling-the-documentation-on-github).
+To enable the code coverage reports, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/codecov/).
+
+## Releasing a new version
+
+- Create an API Token on [Pypi](https://pypi.org/).
+- Add the API Token to your projects secrets with the name `PYPI_TOKEN` by visiting [this page](https://github.com/zawadzkim/gensor/settings/secrets/actions/new).
+- Create a [new release](https://github.com/zawadzkim/gensor/releases/new) on Github.
+- Create a new tag in the form `*.*.*`.
+- For more details, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/cicd/#how-to-trigger-a-release).
+
+---
+
+Repository initiated with [fpgmaas/cookiecutter-poetry](https://github.com/fpgmaas/cookiecutter-poetry).
+

gensor-0.0.1/README.md

@@ -0,0 +1,49 @@
+# gensor
+
+[![Release](https://img.shields.io/github/v/release/zawadzkim/gensor)](https://img.shields.io/github/v/release/zawadzkim/gensor)
+[![Build status](https://img.shields.io/github/actions/workflow/status/zawadzkim/gensor/main.yml?branch=main)](https://github.com/zawadzkim/gensor/actions/workflows/main.yml?query=branch%3Amain)
+[![codecov](https://codecov.io/gh/zawadzkim/gensor/branch/main/graph/badge.svg)](https://codecov.io/gh/zawadzkim/gensor)
+[![Commit activity](https://img.shields.io/github/commit-activity/m/zawadzkim/gensor)](https://img.shields.io/github/commit-activity/m/zawadzkim/gensor)
+[![License](https://img.shields.io/github/license/zawadzkim/gensor)](https://img.shields.io/github/license/zawadzkim/gensor)
+
+Library for handling groundwater sensor data.
+
+- **Github repository**: <https://github.com/zawadzkim/gensor/>
+- **Documentation** <https://zawadzkim.github.io/gensor/>
+
+## Getting started with your project
+
+First, create a repository on GitHub with the same name as this project, and then run the following commands:
+
+```bash
+git init -b main
+git add .
+git commit -m "init commit"
+git remote add origin git@github.com:zawadzkim/gensor.git
+git push -u origin main
+```
+
+Finally, install the environment and the pre-commit hooks with
+
+```bash
+make install
+```
+
+You are now ready to start development on your project!
+The CI/CD pipeline will be triggered when you open a pull request, merge to main, or when you create a new release.
+
+To finalize the set-up for publishing to PyPi or Artifactory, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/publishing/#set-up-for-pypi).
+For activating the automatic documentation with MkDocs, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/mkdocs/#enabling-the-documentation-on-github).
+To enable the code coverage reports, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/codecov/).
+
+## Releasing a new version
+
+- Create an API Token on [Pypi](https://pypi.org/).
+- Add the API Token to your projects secrets with the name `PYPI_TOKEN` by visiting [this page](https://github.com/zawadzkim/gensor/settings/secrets/actions/new).
+- Create a [new release](https://github.com/zawadzkim/gensor/releases/new) on Github.
+- Create a new tag in the form `*.*.*`.
+- For more details, see [here](https://fpgmaas.github.io/cookiecutter-poetry/features/cicd/#how-to-trigger-a-release).
+
+---
+
+Repository initiated with [fpgmaas/cookiecutter-poetry](https://github.com/fpgmaas/cookiecutter-poetry).

gensor-0.0.1/gensor/__init__.py

@@ -0,0 +1,14 @@
+from .compensation import Compensator, compensate
+from .dtypes import Dataset, Timeseries
+from .getters import read_from_csv
+from .preprocessing import OutlierDetection, Transform
+
+__all__ = [
+    "Dataset",
+    "Timeseries",
+    "read_from_csv",
+    "OutlierDetection",
+    "Transform",
+    "Compensator",
+    "compensate",
+]

gensor-0.0.1/gensor/compensation.py

@@ -0,0 +1,110 @@
+"""Compensating the raw data from the absolute pressure transducer to the actual water
+level using the barometric pressure data.
+
+Because van Essen Instrument divers are non-vented pressure transducers, to obtain the
+pressure resulting from the water column above the logger (i.e. the water level), the
+barometric pressure must be subtracted from the raw pressure measurements. In the
+first step the function aligns the two series to the same time step and then subtracts
+the barometric pressure from the raw pressure measurements. For short time periods (when
+for instance a slug test is performed) the barometric pressure can be provided as a
+single float value.
+
+Subsequently the function filters out all records where the absolute water column is
+less than or equal to the cutoff value. This is because when the logger is out of the
+water when the measurement is taken, the absolute water column is close to zero,
+producing erroneous results and spikes in the plots. The cutoff value is set to 5 cm by
+default, but can be adjusted using the cutoff_wc kwarg.
+
+Functions:
+
+    compensate: Compensate raw sensor pressure measurement with barometric pressure.
+"""
+
+from typing import Self
+
+import pydantic as pyd
+
+from .dtypes import Timeseries
+from .exceptions import (
+    InvalidMeasurementTypeError,
+    MissingInputError,
+)
+
+
+class Compensator(pyd.BaseModel):
+    ts: Timeseries
+    barometric: Timeseries | float
+    drop_low_wc: bool = True
+
+    @pyd.field_validator("ts", "barometric", mode="before")
+    def validate_timeseries_type(cls, v):
+        if isinstance(v, Timeseries) and v.variable != "pressure":
+            raise InvalidMeasurementTypeError(v.location)
+        return v
+
+    @pyd.field_validator("ts")
+    def validate_sensor_information(cls, v: Timeseries):
+        if v.sensor is not None and not v.sensor_alt:
+            raise MissingInputError("sensor_alt")
+        return v
+
+    def compensate(self, **kwargs) -> Self | None:
+        """Compensate raw sensor pressure measurement with barometric pressure.
+
+        Parameters:
+            ts (Timeseries): Raw sensor timeseries
+            barometric (Timeseries or float): Barometric pressure timeseries or a single
+                float value. If a float value is provided, it is assumed to be in cmH2O.
+            drop_low_wc (bool): Whether to drop records where the absolute water column is
+                less than or equal to the cutoff value. Defaults to True.
+            inplace (bool): Whether to update the timeseries in place. Defaults to True.
+
+        Keyword Arguments:
+            alignment_period (str): The alignment period for the timeseries.
+                Default is 'H' (hourly).
+            threshold_wc (float): The threshold for the absolute water column.
+                Defaults to 0.5 m.
+
+        Returns:
+            Timeseries: A new Timeseries instance with the compensated data.
+        """
+
+        alignment_period = kwargs.get("alignment_period", "h")
+        threshold_wc = kwargs.get("threshold_wc", 0.5)
+        resample_params = {"freq": alignment_period, "agg_func": "mean"}
+
+        if isinstance(self.barometric, Timeseries):
+            if self.ts == self.barometric:
+                print("Skipping compensation: both timeseries are the same.")
+                return None
+            baro = self.barometric.resample(**resample_params).ts
+        else:
+            baro = self.barometric
+
+        resampled_ts = self.ts.resample(**resample_params)
+
+        # dividing by 100 to convert water column from cmH2O to mH2O
+        watercolumn_ts = resampled_ts.ts.sub(baro).divide(100).dropna()
+
+        if self.drop_low_wc:
+            watercolumn_ts_filtered = watercolumn_ts[
+                watercolumn_ts.abs() > threshold_wc
+            ]
+            print(
+                f"{len(watercolumn_ts) - len(watercolumn_ts_filtered)} records \
+                    dropped due to low water column."
+            )
+            gwl = watercolumn_ts_filtered.add(float(resampled_ts.sensor_alt))
+        else:
+            gwl = watercolumn_ts.add(float(resampled_ts.sensor_alt))
+
+        compensated = resampled_ts.model_copy(
+            update={"ts": gwl, "unit": "m asl", "variable": "head"}
+        )
+
+        return compensated
+
+
+def compensate(ts, barometric, drop_low_wc, **kwargs) -> Timeseries:
+    comp = Compensator(ts=ts, barometric=barometric, drop_low_wc=drop_low_wc)
+    return comp.compensate(**kwargs)

gensor-0.0.1/gensor/db/__init__.py

@@ -0,0 +1,3 @@
+from .connection import DatabaseConnection
+
+__all__ = ["DatabaseConnection"]

gensor-0.0.1/gensor/db/connection.py

@@ -0,0 +1,49 @@
+"""Module for database connection."""
+
+from pathlib import Path
+
+import pydantic as pyd
+from sqlalchemy import Engine, create_engine
+from sqlalchemy.orm import Session, sessionmaker
+
+from ..exceptions import DatabaseNotFound
+
+
+class DatabaseConnection(pyd.BaseModel):
+    """Class for handling the database connection.
+    If no database exists at the specified path, it will be created.
+    If no database is specified, an in-memory database will be used.
+
+    The user should specify the database directory and name separately. If directory is not specified,
+    current directory and a default name are used. ."""
+
+    model_config = pyd.ConfigDict(
+        arbitrary_types_allowed=True, validate_assignment=True
+    )
+
+    in_memory: bool = False
+    db_directory: Path = Path.cwd()
+    db_name: str = "gensor.db"
+    engine: Engine | None = None
+    session: Session | None = None
+
+    def __post_init__(self) -> None:
+        self.connect()
+
+    def _verify_path(self) -> str:
+        if self.in_memory:
+            return "sqlite:///:memory:"
+        else:
+            if not self.db_directory.exists():
+                raise DatabaseNotFound()
+            else:
+                return f"sqlite:///{self.db_directory}/{self.db_name}"
+
+    def connect(self) -> Session:
+        sqlite_path = self._verify_path()
+
+        self.engine = create_engine(sqlite_path)
+        session = sessionmaker(bind=self.engine)
+        self.session = session()
+
+        return session()