shepherd-data 2022.9.3__tar.gz → 2023.3.1__tar.gz

{shepherd_data-2022.9.3 → shepherd_data-2023.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: shepherd_data
-Version: 2022.9.3
+Version: 2023.3.1
 Summary: Programming- and CLI-Interface for the h5-dataformat of the Shepherd-Testbed
 Home-page: https://pypi.org/project/shepherd-data/
 Author: Ingmar Splitt, Kai Geissdoerfer
@@ -20,52 +20,60 @@ Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Information Technology
 Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3.6
 Classifier: Programming Language :: Python :: 3.7
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Natural Language :: English
-Requires-Python: >=3.6
+Requires-Python: >=3.7
 Description-Content-Type: text/markdown
 Provides-Extra: dev
 Provides-Extra: test
 License-File: LICENSE
 
+# Shepherd - Data
+
 [![PyPiVersion](https://img.shields.io/pypi/v/shepherd_data.svg)](https://pypi.org/project/shepherd_data)
 [![Pytest](https://github.com/orgua/shepherd-datalib/actions/workflows/python-app.yml/badge.svg)](https://github.com/orgua/shepherd-datalib/actions/workflows/python-app.yml)
 [![CodeStyle](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
-## Shepherd - Data
-
 This Python Module eases the handling of hdf5-recordings used by the [shepherd](https://github.com/orgua/shepherd)-testbed. Users can read, validate and create files and also extract, down-sample and plot information.
 
-### Installation
+---
 
-#### PIP - Online
+**Main Project**: [https://github.com/orgua/shepherd](https://github.com/orgua/shepherd)
 
-```Shell
+**Source Code**: [https://github.com/orgua/shepherd-datalib](https://github.com/orgua/shepherd-datalib)
+
+---
+
+## Installation
+
+### PIP - Online
+
+```shell
 pip3 install shepherd-data
 ```
 
-#### PIP - Offline
+### PIP - Offline
 
 - clone repository
 - navigate shell into directory
 - install local module
 
-```Shell
+```shell
 git clone https://github.com/orgua/shepherd-datalib
 cd .\shepherd-datalib
 
 pip3 install ./
 ```
 
-### Development
+## Development
 
-#### PipEnv
+### PipEnv
 
 - clone repository
 - navigate shell into directory
@@ -87,20 +95,20 @@ pipenv update
 pipenv install --dev pytest
 ```
 
-#### running Testbench
+### running Testbench
 
 - run pytest
 
-```Shell
+```shell
 pytest
 ```
 
-#### code coverage (with pytest)
+### code coverage (with pytest)
 
 - run coverage
 - check results (in browser `./htmlcov/index.html`)
 
-```Shell
+```shell
 coverage run -m pytest
 
 coverage html
@@ -108,11 +116,11 @@ coverage html
 coverage report
 ```
 
-### Programming Interface
+## Programming Interface
 
-#### Basic Usage (recommendation)
+### Basic Usage (recommendation)
 
-```Python
+```python
 import shepherd_data as shpd
 
 with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
@@ -121,7 +129,7 @@ with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
     print(f"Config: {db.get_config()}")
 ```
 
-#### Available Functionality
+### Available Functionality
 
 - `Reader()`
   - file can be checked for plausibility and validity (`is_valid()`)
@@ -156,7 +164,7 @@ with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
       - 50 Hz measurement with Short-Circuit-Current and two other parameters
       - recorded with "IVonne"
 
-### CLI-Interface
+## CLI-Interface
 
 After installing the module the datalib offers some often needed functionality on the command line:
 
@@ -164,7 +172,7 @@ After installing the module the datalib offers some often needed functionality o
 
 - takes a file or directory as an argument
 
-```Shell
+```shell
 shepherd-data validate dir_or_file
 
 # examples:
@@ -177,7 +185,7 @@ shepherd-data validate hrv_saw_1h.h5
 - takes a file or directory as an argument
 - can take down-sample-factor as an argument
 
-```Shell
+```shell
 shepherd-data extract [-f ds-factor] [-s separator_symbol] dir_or_file
 
 # examples:
@@ -189,7 +197,7 @@ shepherd-data extract -f 1000 -s ; hrv_saw_1h.h5
 
 - takes a file or directory as an argument
 
-```Shell
+```shell
 shepherd-data extract-meta dir_or_file
 
 # examples:
@@ -203,7 +211,7 @@ shepherd-data extract-meta hrv_saw_1h.h5
 - can take start- and end-time as an argument
 - can take image-width and -height as an argument
 
-```Shell
+```shell
 shepherd-data plot [-s start_time] [-e end_time] [-w plot_width] [-h plot_height] [--multiplot] dir_or_file
 
 # examples:
@@ -217,7 +225,7 @@ shepherd-data plot -s10 -e20 hrv_saw_1h.h5
 - takes a file or directory as an argument
 - can take down-sample-factor as an argument
 
-```Shell
+```shell
 shepherd-data downsample [-f ds-factor] [-r sample-rate] dir_or_file
 
 # examples:
@@ -226,7 +234,7 @@ shepherd-data downsample -f 1000 hrv_saw_1h.h5
 shepherd-data downsample -r 100 hrv_saw_1h.h5
 ```
 
-### Data-Layout and Design choices
+## Data-Layout and Design choices
 
 Details about the file-structure can be found in the [main-project](https://github.com/orgua/shepherd/blob/main/docs/user/data_format.rst).
 
@@ -234,7 +242,7 @@ TODO:
 - update design of file
 - data dtype, mode, ...
 
-#### Modes and Datatypes
+### Modes and Datatypes
 
 - Mode `harvester` recorded a harvesting-source like solar with one of various algorithms
   - Datatype `ivsample` is directly usable by shepherd, input for virtual source / converter
@@ -243,7 +251,7 @@ TODO:
 - Mode `emulator` replayed a harvester-recording through a virtual converter and supplied a target while recording the power-consumption
   - Datatype `ivsample` is the only output of this mode
 
-#### Compression & Beaglebone
+### Compression & Beaglebone
 
 - supported are uncompressed, lzf and gzip with level 1 (order of recommendation)
   - lzf seems better-suited due to lower load, or if space isn't a constraint: uncompressed (None as argument)
@@ -268,15 +276,23 @@ TODO:
   emu_120s_unc_to_unc.h5 	-> emulator, cpu_util [%] = 51.69, data-rate = 1564.0 KiB/s
 ```
 
-### Release-Procedure
+## Release-Procedure
 
 - increase version number in __init__.py
+- install and run pre-commit, for QA-Checks, see steps below
 - every commit get automatically tested by Github
 - put together a release on Github - the tag should match current version-number
 - Github automatically pushes release to pypi
 
-### Open Tasks
+```shell
+pip3 install pre-commit
+
+pre-commit run --all-files
+```
+
+## Open Tasks
 
+- [click progressbar](https://click.palletsprojects.com/en/8.1.x/api/#click.progressbar) -> could replace tqdm
 - implementations for this lib
   - generalize up- and down-sampling, use out_sample_rate instead of ds-factor
     - lib samplerate (tested) -> promising, but designed for float32 and range of +-1.0

{shepherd_data-2022.9.3 → shepherd_data-2023.3.1}/README.md

@@ -1,35 +1,43 @@
+# Shepherd - Data
+
 [![PyPiVersion](https://img.shields.io/pypi/v/shepherd_data.svg)](https://pypi.org/project/shepherd_data)
 [![Pytest](https://github.com/orgua/shepherd-datalib/actions/workflows/python-app.yml/badge.svg)](https://github.com/orgua/shepherd-datalib/actions/workflows/python-app.yml)
 [![CodeStyle](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
-## Shepherd - Data
-
 This Python Module eases the handling of hdf5-recordings used by the [shepherd](https://github.com/orgua/shepherd)-testbed. Users can read, validate and create files and also extract, down-sample and plot information.
 
-### Installation
+---
 
-#### PIP - Online
+**Main Project**: [https://github.com/orgua/shepherd](https://github.com/orgua/shepherd)
 
-```Shell
+**Source Code**: [https://github.com/orgua/shepherd-datalib](https://github.com/orgua/shepherd-datalib)
+
+---
+
+## Installation
+
+### PIP - Online
+
+```shell
 pip3 install shepherd-data
 ```
 
-#### PIP - Offline
+### PIP - Offline
 
 - clone repository
 - navigate shell into directory
 - install local module
 
-```Shell
+```shell
 git clone https://github.com/orgua/shepherd-datalib
 cd .\shepherd-datalib
 
 pip3 install ./
 ```
 
-### Development
+## Development
 
-#### PipEnv
+### PipEnv
 
 - clone repository
 - navigate shell into directory
@@ -51,20 +59,20 @@ pipenv update
 pipenv install --dev pytest
 ```
 
-#### running Testbench
+### running Testbench
 
 - run pytest
 
-```Shell
+```shell
 pytest
 ```
 
-#### code coverage (with pytest)
+### code coverage (with pytest)
 
 - run coverage
 - check results (in browser `./htmlcov/index.html`)
 
-```Shell
+```shell
 coverage run -m pytest
 
 coverage html
@@ -72,11 +80,11 @@ coverage html
 coverage report
 ```
 
-### Programming Interface
+## Programming Interface
 
-#### Basic Usage (recommendation)
+### Basic Usage (recommendation)
 
-```Python
+```python
 import shepherd_data as shpd
 
 with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
@@ -85,7 +93,7 @@ with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
     print(f"Config: {db.get_config()}")
 ```
 
-#### Available Functionality
+### Available Functionality
 
 - `Reader()`
   - file can be checked for plausibility and validity (`is_valid()`)
@@ -120,7 +128,7 @@ with shpd.Reader("./hrv_sawtooth_1h.h5") as db:
       - 50 Hz measurement with Short-Circuit-Current and two other parameters
       - recorded with "IVonne"
 
-### CLI-Interface
+## CLI-Interface
 
 After installing the module the datalib offers some often needed functionality on the command line:
 
@@ -128,7 +136,7 @@ After installing the module the datalib offers some often needed functionality o
 
 - takes a file or directory as an argument
 
-```Shell
+```shell
 shepherd-data validate dir_or_file
 
 # examples:
@@ -141,7 +149,7 @@ shepherd-data validate hrv_saw_1h.h5
 - takes a file or directory as an argument
 - can take down-sample-factor as an argument
 
-```Shell
+```shell
 shepherd-data extract [-f ds-factor] [-s separator_symbol] dir_or_file
 
 # examples:
@@ -153,7 +161,7 @@ shepherd-data extract -f 1000 -s ; hrv_saw_1h.h5
 
 - takes a file or directory as an argument
 
-```Shell
+```shell
 shepherd-data extract-meta dir_or_file
 
 # examples:
@@ -167,7 +175,7 @@ shepherd-data extract-meta hrv_saw_1h.h5
 - can take start- and end-time as an argument
 - can take image-width and -height as an argument
 
-```Shell
+```shell
 shepherd-data plot [-s start_time] [-e end_time] [-w plot_width] [-h plot_height] [--multiplot] dir_or_file
 
 # examples:
@@ -181,7 +189,7 @@ shepherd-data plot -s10 -e20 hrv_saw_1h.h5
 - takes a file or directory as an argument
 - can take down-sample-factor as an argument
 
-```Shell
+```shell
 shepherd-data downsample [-f ds-factor] [-r sample-rate] dir_or_file
 
 # examples:
@@ -190,7 +198,7 @@ shepherd-data downsample -f 1000 hrv_saw_1h.h5
 shepherd-data downsample -r 100 hrv_saw_1h.h5
 ```
 
-### Data-Layout and Design choices
+## Data-Layout and Design choices
 
 Details about the file-structure can be found in the [main-project](https://github.com/orgua/shepherd/blob/main/docs/user/data_format.rst).
 
@@ -198,7 +206,7 @@ TODO:
 - update design of file
 - data dtype, mode, ...
 
-#### Modes and Datatypes
+### Modes and Datatypes
 
 - Mode `harvester` recorded a harvesting-source like solar with one of various algorithms
   - Datatype `ivsample` is directly usable by shepherd, input for virtual source / converter
@@ -207,7 +215,7 @@ TODO:
 - Mode `emulator` replayed a harvester-recording through a virtual converter and supplied a target while recording the power-consumption
   - Datatype `ivsample` is the only output of this mode
 
-#### Compression & Beaglebone
+### Compression & Beaglebone
 
 - supported are uncompressed, lzf and gzip with level 1 (order of recommendation)
   - lzf seems better-suited due to lower load, or if space isn't a constraint: uncompressed (None as argument)
@@ -232,15 +240,23 @@ TODO:
   emu_120s_unc_to_unc.h5 	-> emulator, cpu_util [%] = 51.69, data-rate = 1564.0 KiB/s
 ```
 
-### Release-Procedure
+## Release-Procedure
 
 - increase version number in __init__.py
+- install and run pre-commit, for QA-Checks, see steps below
 - every commit get automatically tested by Github
 - put together a release on Github - the tag should match current version-number
 - Github automatically pushes release to pypi
 
-### Open Tasks
+```shell
+pip3 install pre-commit
+
+pre-commit run --all-files
+```
+
+## Open Tasks
 
+- [click progressbar](https://click.palletsprojects.com/en/8.1.x/api/#click.progressbar) -> could replace tqdm
 - implementations for this lib
   - generalize up- and down-sampling, use out_sample_rate instead of ds-factor
     - lib samplerate (tested) -> promising, but designed for float32 and range of +-1.0

shepherd_data-2023.3.1/pyproject.toml

@@ -0,0 +1,115 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[tool.pyright]
+root = "./"
+include = ['./src/shepherd_data', './src/examples', "./tests"]
+pythonVersion = "3.8"
+pythonPlatform = "All"
+reportMissingParameterType = true
+
+[tool.codespell]
+builtin = "clear,rare,informal,usage,code,en-GB_to_en-US"
+
+#skip = ""
+#ignore-words-list = ""
+
+# options without argument
+check-filenames = ""
+# check-hidden = ""
+
+[tool.isort]
+profile = "black"
+filter_files = true
+force_single_line = true
+
+[tool.flake8]
+max-line-length = 100
+require-plugins = [
+    # https://github.com/DmytroLitvinov/awesome-flake8-extensions
+
+    ### Bugs & Code-Smells
+    "flake8-bugbear",
+    "flake8-secure-coding-standard",
+    "flake8-bandit",
+    "flake8-builtins",
+
+    ### Clean Code
+    "flake8-comprehensions",
+    "flake8-simplify",
+    "flake8-eradicate",
+    "flake8-commas",
+
+    ### Limitations
+    "flake8-blind-except",
+    "flake8-logging-format",
+    "flake8-print",
+
+    ### Documentation
+    "flake8-comments",
+    # flake8-docstrings
+    # flake8-rst-docstrings
+
+    ### Test-Improvements
+    "flake8-assertive",
+]
+
+extend-ignore = [
+    # black-formatter-specific
+    "E203",
+    # open() vs os.open()
+    "SCS109",
+    # hideous commas
+    "C812",
+    "C815",
+    "C813",
+    # DOCString TODO: reduce here
+    "D100",
+    "D102",
+    "D103",
+]
+
+per-file-ignores = """
+# asserts (in tests are ok), also allow print() instead of logger
+tests/*: SCS108, S101, T201
+# allow print()
+src/examples/*: T201
+"""
+
+count = true
+statistics = true
+
+[tool.ruff]  # TODO: once stable it replaces flake8, pyupgrade, isort, ...
+line-length = 100
+select = [
+    "A",
+    "ARG",
+    "B",
+    "C",
+    "C4",
+    "E",
+    "ERA",
+    "F",
+#    "I",  # incomplete isort
+    "N",
+    "PLC",
+    "PLE",
+    "PLR",
+    "PLW",
+    "RET",
+    "S",
+    "T10",
+    "UP",
+    "W",  # TODO: just a proposed starting-config
+]
+ignore = ["A003"]
+target-version = "py38"
+
+[tool.ruff.per-file-ignores]
+"__init__.py" = ["F401"]
+"tests/**" = ["ARG", "S"]
+
+[tool.ruff.mccabe]
+# Unlike Flake8, default to a complexity level of 10.
+max-complexity = 10

{shepherd_data-2022.9.3 → shepherd_data-2023.3.1}/setup.cfg

@@ -20,11 +20,11 @@ classifiers =
 	Intended Audience :: Developers
 	Intended Audience :: Information Technology
 	Intended Audience :: Science/Research
-	Programming Language :: Python :: 3.6
 	Programming Language :: Python :: 3.7
 	Programming Language :: Python :: 3.8
 	Programming Language :: Python :: 3.9
 	Programming Language :: Python :: 3.10
+	Programming Language :: Python :: 3.11
 	License :: OSI Approved :: MIT License
 	Operating System :: OS Independent
 	Natural Language :: English
@@ -35,16 +35,16 @@ package_dir =
 	=src
 zip_safe = True
 include_package_data = True
-python_requires = >= 3.6
+python_requires = >= 3.7
 install_requires = 
 	click
 	h5py
-	matplotlib
+	matplotlib  # full-version
 	numpy
-	pandas
+	pandas  # full-version
 	pyYAML
-	scipy
-	tqdm
+	scipy   # full-version
+	tqdm    # full-version
 
 [options.extras_require]
 dev = 
@@ -53,6 +53,8 @@ dev =
 	flake8
 	twine
 	pre-commit
+	pyright
+	pandas-stubs
 test = 
 	pytest
 	pytest-click
@@ -60,7 +62,7 @@ test =
 
 [options.entry_points]
 console_scripts = 
-	shepherd_data = shepherd_data.cli:cli
+	shepherd-data = shepherd_data.cli:cli
 
 [options.packages.find]
 where = src
@@ -77,51 +79,6 @@ test = pytest
 [tool:pytest]
 addopts = -vvv
 
-[codespell]
-builtin = clear,rare,informal,usage,code,en-GB_to_en-US
-check-filenames = 
-check-hidden = 
-
-[isort]
-profile = black
-force_single_line = true
-
-[flake8]
-max-line-length = 100
-require-plugins = 
-	
-	flake8-bugbear
-	flake8-secure-coding-standard
-	flake8-bandit
-	flake8-builtins
-	
-	flake8-comprehensions
-	flake8-simplify
-	flake8-eradicate
-	flake8-commas
-	
-	flake8-blind-except
-	flake8-logging-format
-	flake8-print
-	
-	flake8-comments
-	
-	flake8-assertive
-extend-ignore = 
-	E203
-	SCS109
-	C812
-	C815
-	C813
-	D100
-	D102
-	D103
-per-file-ignores = 
-	tests/*: SCS108 S101 T201
-	src/examples/*: T201
-count = True
-statistics = True
-
 [egg_info]
 tag_build = 
 tag_date = 0

{shepherd_data-2022.9.3 → shepherd_data-2023.3.1}/src/shepherd_data/__init__.py

@@ -10,7 +10,8 @@ import logging
 from .reader import Reader
 from .writer import Writer
 
-__version__ = "2022.9.3"
+__version__ = "2023.3.1"
+
 __all__ = [
     "Reader",
     "Writer",