shepherd-core 2025.5.2__tar.gz → 2025.6.1__tar.gz

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: shepherd_core
-Version: 2025.5.2
+Version: 2025.6.1
 Summary: Programming- and CLI-Interface for the h5-dataformat of the Shepherd-Testbed
 Author-email: Ingmar Splitt <ingmar.splitt@tu-dresden.de>
 Maintainer-email: Ingmar Splitt <ingmar.splitt@tu-dresden.de>
-Project-URL: Documentation, https://github.com/orgua/shepherd-datalib/blob/main/README.md
-Project-URL: Issues, https://github.com/orgua/shepherd-datalib/issues
+Project-URL: Documentation, https://github.com/nes-lab/shepherd-tools/blob/main/README.md
+Project-URL: Issues, https://github.com/nes-lab/shepherd-tools/issues
 Project-URL: Source, https://pypi.org/project/shepherd-core/
 Keywords: testbed,beaglebone,pru,batteryless,energyharvesting,solar
 Platform: unix
@@ -56,16 +56,16 @@ Requires-Dist: coverage; extra == "test"
 
 # Core Library
 
-[![PyPiVersion](https://img.shields.io/pypi/v/shepherd_core.svg)](https://pypi.org/project/shepherd_core)
+[![PyPIVersion](https://img.shields.io/pypi/v/shepherd_core.svg)](https://pypi.org/project/shepherd_core)
 [![image](https://img.shields.io/pypi/pyversions/shepherd_core.svg)](https://pypi.python.org/pypi/shepherd-core)
-[![Pytest](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml/badge.svg)](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml)
+[![QA-Tests](https://github.com/nes-lab/shepherd-tools/actions/workflows/quality_assurance.yaml/badge.svg)](https://github.com/nes-lab/shepherd-tools/actions/workflows/quality_assurance.yaml)
 [![CodeStyle](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 
-**Main Documentation**: <https://orgua.github.io/shepherd>
+**Main Documentation**: <https://nes-lab.github.io/shepherd>
 
-**Source Code**: <https://github.com/orgua/shepherd-datalib>
+**Source Code**: <https://github.com/nes-lab/shepherd-tools>
 
-**Main Project**: <https://github.com/orgua/shepherd>
+**Main Project**: <https://github.com/nes-lab/shepherd>
 
 ---
 
@@ -87,7 +87,7 @@ For postprocessing shepherds .h5-files usage of [shepherd_data](https://pypi.org
 - decode waveforms (gpio-state & timestamp) to UART
 - create an inventory (for deployed versions of software, hardware)
 
-See [official documentation](https://orgua.github.io/shepherd) or [example scripts](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/orgua/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.
+See [official documentation](https://nes-lab.github.io/shepherd) or [example scripts](https://github.com/nes-lab/shepherd-tools/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/nes-lab/shepherd-tools/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/nes-lab/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.
 
 ## Config-Models in Detail
 
@@ -145,9 +145,9 @@ pip install shepherd-data -U
 For bleeding-edge-features or dev-work it is possible to install directly from GitHub-Sources (here `dev`-branch):
 
 ```Shell
-pip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U
+pip install git+https://github.com/nes-lab/shepherd-tools.git@dev#subdirectory=shepherd_core -U
 # and on sheep with newer debian
-sudo pip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U --break-system-packages
+sudo pip install git+https://github.com/nes-lab/shepherd-tools.git@dev#subdirectory=shepherd_core -U --break-system-packages
 ```
 
 If you are working with ``.elf``-files (embedding into experiments) you make "objcopy" accessible to python. In Ubuntu, you can either install ``build-essential`` or ``binutils-$ARCH`` with arch being ``msp430`` or ``arm-none-eabi`` for the nRF52.
@@ -179,7 +179,7 @@ To run the testbench, follow these steps:
 3. run the testbench (~ 320 tests):
 
 ```Shell
-cd shepherd-datalib/shepherd_core
+cd shepherd-tools/shepherd_core
 pip3 install ./[tests]
 pytest
 ```

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/README.md

@@ -1,15 +1,15 @@
 # Core Library
 
-[![PyPiVersion](https://img.shields.io/pypi/v/shepherd_core.svg)](https://pypi.org/project/shepherd_core)
+[![PyPIVersion](https://img.shields.io/pypi/v/shepherd_core.svg)](https://pypi.org/project/shepherd_core)
 [![image](https://img.shields.io/pypi/pyversions/shepherd_core.svg)](https://pypi.python.org/pypi/shepherd-core)
-[![Pytest](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml/badge.svg)](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml)
+[![QA-Tests](https://github.com/nes-lab/shepherd-tools/actions/workflows/quality_assurance.yaml/badge.svg)](https://github.com/nes-lab/shepherd-tools/actions/workflows/quality_assurance.yaml)
 [![CodeStyle](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 
-**Main Documentation**: <https://orgua.github.io/shepherd>
+**Main Documentation**: <https://nes-lab.github.io/shepherd>
 
-**Source Code**: <https://github.com/orgua/shepherd-datalib>
+**Source Code**: <https://github.com/nes-lab/shepherd-tools>
 
-**Main Project**: <https://github.com/orgua/shepherd>
+**Main Project**: <https://github.com/nes-lab/shepherd>
 
 ---
 
@@ -31,7 +31,7 @@ For postprocessing shepherds .h5-files usage of [shepherd_data](https://pypi.org
 - decode waveforms (gpio-state & timestamp) to UART
 - create an inventory (for deployed versions of software, hardware)
 
-See [official documentation](https://orgua.github.io/shepherd) or [example scripts](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/orgua/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.
+See [official documentation](https://nes-lab.github.io/shepherd) or [example scripts](https://github.com/nes-lab/shepherd-tools/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/nes-lab/shepherd-tools/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/nes-lab/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.
 
 ## Config-Models in Detail
 
@@ -89,9 +89,9 @@ pip install shepherd-data -U
 For bleeding-edge-features or dev-work it is possible to install directly from GitHub-Sources (here `dev`-branch):
 
 ```Shell
-pip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U
+pip install git+https://github.com/nes-lab/shepherd-tools.git@dev#subdirectory=shepherd_core -U
 # and on sheep with newer debian
-sudo pip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U --break-system-packages
+sudo pip install git+https://github.com/nes-lab/shepherd-tools.git@dev#subdirectory=shepherd_core -U --break-system-packages
 ```
 
 If you are working with ``.elf``-files (embedding into experiments) you make "objcopy" accessible to python. In Ubuntu, you can either install ``build-essential`` or ``binutils-$ARCH`` with arch being ``msp430`` or ``arm-none-eabi`` for the nRF52.
@@ -123,7 +123,7 @@ To run the testbench, follow these steps:
 3. run the testbench (~ 320 tests):
 
 ```Shell
-cd shepherd-datalib/shepherd_core
+cd shepherd-tools/shepherd_core
 pip3 install ./[tests]
 pytest
 ```

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/examples/experiment_generic_var1.py

@@ -48,7 +48,7 @@ xp = sm.Experiment(
             ),
             power_tracing=None,
             gpio_tracing=sm.GpioTracing(),
-            uart_tracing=sm.UartTracing(baudrate=115_200),
+            uart_logging=sm.UartLogging(baudrate=115_200),
         ),
     ],
 )

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/pyproject.toml

@@ -71,8 +71,8 @@ file = "README.md"
 content-type = "text/markdown"
 
 [project.urls]
-Documentation = "https://github.com/orgua/shepherd-datalib/blob/main/README.md"
-Issues = "https://github.com/orgua/shepherd-datalib/issues"
+Documentation = "https://github.com/nes-lab/shepherd-tools/blob/main/README.md"
+Issues = "https://github.com/nes-lab/shepherd-tools/issues"
 Source = "https://pypi.org/project/shepherd-core/"
 
 [build-system]

shepherd_core-2025.6.1/shepherd_core/commons.py

@@ -0,0 +1,6 @@
+"""Container for commonly shared constants."""
+
+from .config import config
+
+# TODO: deprecated - replace with config in subprojects and then remove here
+SAMPLERATE_SPS_DEFAULT: int = config.SAMPLERATE_SPS

shepherd_core-2025.6.1/shepherd_core/config.py

@@ -0,0 +1,34 @@
+"""Container for a common configuration.
+
+This can be adapted by the user by importing 'config' and changing its variables.
+"""
+
+from pydantic import BaseModel
+from pydantic import HttpUrl
+
+
+class ConfigDefault(BaseModel):
+    """Container for a common configuration."""
+
+    __slots__ = ()
+
+    TESTBED: str = "shepherd_tud_nes"
+    """name of the testbed to validate against - if enabled - see switch below"""
+    VALIDATE_INFRA: bool = True
+    """switch to turn on / off deep validation of data models also considering the current
+    layout & infrastructure of the testbed.
+    """
+
+    SAMPLERATE_SPS: int = 100_000
+    """Rate of IV-Recording of the testbed."""
+
+    UID_NAME: str = "SHEPHERD_NODE_ID"
+    """Variable name to patch in ELF-file"""
+    UID_SIZE: int = 2
+    """Variable size in Byte"""
+
+    TESTBED_SERVER: HttpUrl = "https://shepherd.cfaed.tu-dresden.de:8000/"
+    """Server that holds up to date testbed fixtures"""
+
+
+config = ConfigDefault()

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/__init__.py

@@ -31,7 +31,7 @@ from .experiment.observer_features import GpioLevel
 from .experiment.observer_features import GpioTracing
 from .experiment.observer_features import PowerTracing
 from .experiment.observer_features import SystemLogging
-from .experiment.observer_features import UartTracing
+from .experiment.observer_features import UartLogging
 from .experiment.target_config import TargetConfig
 
 __all__ = [
@@ -55,7 +55,7 @@ __all__ = [
     "ShpModel",
     "SystemLogging",
     "TargetConfig",
-    "UartTracing",
+    "UartLogging",
     "VirtualHarvesterConfig",
     "VirtualSourceConfig",
     "Wrapper",

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/base/calibration.py

@@ -95,14 +95,19 @@ cal_hrv_legacy = {  # legacy translator
     "adc_voltage": "adc_V_Sense",  # datalog voltage
 }
 
+# defaults (pre-init complex types)
+cal_pair_dac_V = CalibrationPair.from_fn(dac_voltage_to_raw, unit="V")
+cal_pair_adc_V = CalibrationPair.from_fn(adc_voltage_to_raw, unit="V")
+cal_pair_adc_C = CalibrationPair.from_fn(adc_current_to_raw, unit="A")
+
 
 class CalibrationHarvester(ShpModel):
     """Container for all calibration-pairs for that device."""
 
-    dac_V_Hrv: CalibrationPair = CalibrationPair.from_fn(dac_voltage_to_raw, unit="V")
-    dac_V_Sim: CalibrationPair = CalibrationPair.from_fn(dac_voltage_to_raw, unit="V")
-    adc_V_Sense: CalibrationPair = CalibrationPair.from_fn(adc_voltage_to_raw, unit="V")
-    adc_C_Hrv: CalibrationPair = CalibrationPair.from_fn(adc_current_to_raw, unit="A")
+    dac_V_Hrv: CalibrationPair = cal_pair_dac_V
+    dac_V_Sim: CalibrationPair = cal_pair_dac_V
+    adc_V_Sense: CalibrationPair = cal_pair_adc_V
+    adc_C_Hrv: CalibrationPair = cal_pair_adc_C
 
     def export_for_sysfs(self) -> dict:
         """Convert and write the essential data.
@@ -143,10 +148,10 @@ class CalibrationEmulator(ShpModel):
     Differentiates between both target-ports A/B.
     """
 
-    dac_V_A: CalibrationPair = CalibrationPair.from_fn(dac_voltage_to_raw, unit="V")
-    dac_V_B: CalibrationPair = CalibrationPair.from_fn(dac_voltage_to_raw, unit="V")
-    adc_C_A: CalibrationPair = CalibrationPair.from_fn(adc_current_to_raw, unit="A")
-    adc_C_B: CalibrationPair = CalibrationPair.from_fn(adc_current_to_raw, unit="A")
+    dac_V_A: CalibrationPair = cal_pair_dac_V
+    dac_V_B: CalibrationPair = cal_pair_dac_V
+    adc_C_A: CalibrationPair = cal_pair_adc_C
+    adc_C_B: CalibrationPair = cal_pair_adc_C
 
     def export_for_sysfs(self) -> dict:
         """Convert and write the essential data.

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/base/content.py

@@ -1,21 +1,16 @@
 """Base-Model for all content."""
 
-import hashlib
 from datetime import datetime
 from typing import Annotated
 from typing import Optional
-from typing import Union
 from uuid import uuid4
 
-from pydantic import UUID4
 from pydantic import Field
 from pydantic import StringConstraints
 from pydantic import model_validator
 from typing_extensions import Self
-from typing_extensions import deprecated
 
 from .shepherd import ShpModel
-from .timezone import local_now
 
 # constr -> to_lower=True, max_length=16, regex=r"^[\w]+$"
 # ⤷ Regex = AlphaNum
@@ -26,24 +21,20 @@ SafeStr = Annotated[str, StringConstraints(pattern=r"^[ -~]+$")]
 # ⤷ Regex = All Printable ASCII-Characters with Space
 
 
-@deprecated("use UUID4 instead")
 def id_default() -> int:
     """Generate a unique ID - usable as default value.
 
-    Note: IdInt has space for 128 bit, so 128/4 = 32 hex-chars
+    Note: IdInt in mongoDB can currently handle 8 bytes (16 hex-values)
     """
-    time_stamp = str(local_now()).encode("utf-8")
-    time_hash = hashlib.sha3_224(time_stamp).hexdigest()[-32:]
-    return int(time_hash, 16)
+    return int(uuid4().hex[-15:], 16)
 
 
 class ContentModel(ShpModel):
     """Base-Model for content with generalized properties."""
 
-    # id: UUID4 = Field(  # TODO: db-migration - temp fix for documentation
-    id: Union[UUID4, int] = Field(
+    id: int = Field(
         description="Unique ID",
-        default_factory=uuid4,
+        default_factory=id_default,
     )
     name: NameStr
     description: Annotated[Optional[SafeStr], Field(description="Required when public")] = None

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/base/wrapper.py

@@ -17,11 +17,11 @@ class Wrapper(BaseModel):
     """Generalized web- & file-interface for all models with dynamic typecasting."""
 
     datatype: str
-    # ⤷ model-name
+    """ ⤷ model-name"""
     comment: Optional[SafeStrClone] = None
     created: Optional[datetime] = None
-    # ⤷ Optional metadata
+    """ ⤷ Optional metadata"""
     lib_ver: Optional[str] = version
-    # ⤷ for debug-purposes and later comp-checks
+    """ ⤷ for debug-purposes and later compatibility-checks"""
     parameters: dict
-    # ⤷ ShpModel
+    """ ⤷ ShpModel"""

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/content/_external_fixtures.yaml

@@ -1,7 +1,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.756999+02:00
   parameters:
-    id: 2639560972524229652
+    id: 263956097252422
     name: eenv_static_2000mV_10mA_3600s
     description: Artificial static Energy Environment, 2000mV, 10mA, 3600s
     comment: null
@@ -24,7 +24,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.764595+02:00
   parameters:
-    id: 9823394105967169626
+    id: 98233941059
     name: eenv_static_2000mV_1mA_3600s
     description: Artificial static Energy Environment, 2000mV, 1mA, 3600s
     comment: null
@@ -47,7 +47,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.772144+02:00
   parameters:
-    id: 3900615675169501222
+    id: 39006156751
     name: eenv_static_2000mV_50mA_3600s
     description: Artificial static Energy Environment, 2000mV, 50mA, 3600s
     comment: null
@@ -70,7 +70,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.779737+02:00
   parameters:
-    id: 14796673729431137386
+    id: 1479667372943113
     name: eenv_static_2000mV_5mA_3600s
     description: Artificial static Energy Environment, 2000mV, 5mA, 3600s
     comment: null
@@ -93,7 +93,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.787329+02:00
   parameters:
-    id: 6648482606607441403
+    id: 664848260660744
     name: eenv_static_3000mV_10mA_3600s
     description: Artificial static Energy Environment, 3000mV, 10mA, 3600s
     comment: null
@@ -116,7 +116,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.794922+02:00
   parameters:
-    id: 13566000951043177991
+    id: 1356600095104317
     name: eenv_static_3000mV_1mA_3600s
     description: Artificial static Energy Environment, 3000mV, 1mA, 3600s
     comment: null
@@ -139,7 +139,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.802410+02:00
   parameters:
-    id: 7977778327156610158
+    id: 797777832715661
     name: eenv_static_3000mV_50mA_3600s
     description: Artificial static Energy Environment, 3000mV, 50mA, 3600s
     comment: null
@@ -162,7 +162,7 @@
 - datatype: EnergyEnvironment
   created: 2025-05-12 17:28:39.810147+02:00
   parameters:
-    id: 4900162978999238419
+    id: 490016297899923
     name: eenv_static_3000mV_5mA_3600s
     description: Artificial static Energy Environment, 3000mV, 5mA, 3600s
     comment: null
@@ -269,7 +269,7 @@
 - datatype: Firmware
   created: 2025-05-12 17:28:39.851412+02:00
   parameters:
-    id: 7163917825449888392
+    id: 716391782544988
     name: nrf52_deep_sleep
     description: practically turned off MCU with the lowest possible consumption
     comment: null
@@ -325,7 +325,7 @@
 - datatype: Firmware
   created: 2025-05-12 17:28:39.868340+02:00
   parameters:
-    id: 3174430733058172825
+    id: 317443073305817
     name: nrf52_rf_survey
     description: Link Matrix Generator - TX-Unit - sends packet with every possible
       P_TX, loops until stopped
@@ -354,7 +354,7 @@
 - datatype: Firmware
   created: 2025-05-12 17:28:39.876819+02:00
   parameters:
-    id: 16381936580724580968
+    id: 1638193658072458
     name: nrf52_rf_test
     description: sends out 1 BLE-Packet per second (verify with an app like 'RaMBLE')
     comment: null

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/content/energy_environment.py

@@ -28,7 +28,7 @@ class EnergyEnvironment(ContentModel):
     data_path: Path
     data_type: EnergyDType
     data_local: bool = True
-    # ⤷ signals that file has to be copied to testbed
+    """ ⤷ signals that file has to be copied to testbed"""
 
     duration: PositiveFloat
     energy_Ws: PositiveFloat

{shepherd_core-2025.5.2 → shepherd_core-2025.6.1}/shepherd_core/data_models/content/firmware.py

@@ -62,7 +62,7 @@ class Firmware(ContentModel, title="Firmware of Target"):
     data_type: FirmwareDType
     data_hash: Optional[str] = None
     data_local: bool = True
-    # ⤷ signals that file has to be copied to testbed
+    """ ⤷ signals that file has to be copied to testbed"""
 
     @model_validator(mode="before")
     @classmethod
@@ -125,12 +125,17 @@ class Firmware(ContentModel, title="Firmware of Target"):
             if "mcu" not in kwargs:
                 kwargs["mcu"] = arch_to_mcu[arch]
 
+        # verification of ELF - warn if something is off
+        # -> adds ARCH if it is able to derive
         if kwargs["data_type"] == FirmwareDType.base64_elf:
             arch = fw_tools.read_arch(file)
-            if "msp430" in arch and not fw_tools.is_elf_msp430(file):
-                raise ValueError("File is not a ELF for msp430")
-            if ("nrf52" in arch or "arm" in arch) and not fw_tools.is_elf_nrf52(file):
-                raise ValueError("File is not a ELF for nRF52")
+            try:
+                if "msp430" in arch and not fw_tools.is_elf_msp430(file):
+                    raise ValueError("File is not a ELF for msp430")
+                if ("nrf52" in arch or "arm" in arch) and not fw_tools.is_elf_nrf52(file):
+                    raise ValueError("File is not a ELF for nRF52")
+            except RuntimeError:
+                logger.warning("ObjCopy not found -> Arch of Firmware can't be verified")
             logger.debug("ELF-File '%s' has arch: %s", file.name, arch)
             if "mcu" not in kwargs:
                 kwargs["mcu"] = arch_to_mcu[arch]