openstef 3.2.16__tar.gz → 3.4.16__tar.gz

{openstef-3.2.16/openstef.egg-info → openstef-3.4.16}/PKG-INFO

@@ -1,27 +1,41 @@
 Metadata-Version: 2.1
 Name: openstef
-Version: 3.2.16
+Version: 3.4.16
 Summary: Open short term energy forecaster
 Home-page: https://github.com/OpenSTEF/openstef
 Author: Alliander N.V
 Author-email: korte.termijn.prognoses@alliander.com
 License: MPL-2.0
 Keywords: energy,forecasting,machinelearning
-Platform: UNKNOWN
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
-Classifier: Programming Language :: Python :: 3.7
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
-Requires-Python: >=3.7.0
+Requires-Python: >=3.9.0
 Description-Content-Type: text/markdown
-Provides-Extra: proloaf
 License-File: LICENSE
+Requires-Dist: holidays==0.21
+Requires-Dist: joblib==1.3.2
+Requires-Dist: lightgbm~=3.3
+Requires-Dist: matplotlib~=3.7
+Requires-Dist: mlflow~=2.3
+Requires-Dist: networkx~=3.1
+Requires-Dist: optuna~=3.1
+Requires-Dist: optuna-integration~=3.6
+Requires-Dist: pandas~=2.2.0
+Requires-Dist: plotly~=5.18
+Requires-Dist: pvlib==0.9.4
+Requires-Dist: pydantic~=2.4
+Requires-Dist: pymsteams~=0.2.2
+Requires-Dist: scikit-learn~=1.3
+Requires-Dist: scipy~=1.10
+Requires-Dist: statsmodels~=0.13.5
+Requires-Dist: structlog<25,>=23.1
+Requires-Dist: xgboost~=2.0
 
 <!--
-SPDX-FileCopyrightText: 2017-2022 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
 
 SPDX-License-Identifier: MPL-2.0
 -->
@@ -40,10 +54,37 @@ SPDX-License-Identifier: MPL-2.0
 [![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=OpenSTEF_openstef&metric=sqale_index)](https://sonarcloud.io/dashboard?id=OpenSTEF_openstef)
 [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=OpenSTEF_openstef&metric=vulnerabilities)](https://sonarcloud.io/dashboard?id=OpenSTEF_openstef)
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5585/badge)](https://bestpractices.coreinfrastructure.org/projects/5585)
+[![Downloads](https://static.pepy.tech/badge/openstef)](https://pepy.tech/project/openstef)
+[![Downloads](https://static.pepy.tech/badge/openstef/month)](https://pepy.tech/project/openstef)
 
 # OpenSTEF
 
-OpenSTEF is a Python package which is used to make short term forecasts for the energy sector. This repository contains all components for the machine learning pipeline required to make a forecast. In order to use the package you need to provide your own data storage and retrieval interface. `openstef` is available at: https://pypi.org/project/openstef/
+OpenSTEF is a Python package designed for generating short-term forecasts in the energy sector. The repository includes all the essential components required for machine learning pipelines that facilitate the forecasting process. To utilize the package, users are required to furnish their own data storage and retrieval interface.
+
+# Table of contents
+- [OpenSTEF](#openstef)
+- [Table of contents](#table-of-contents)
+- [External information sources](#external-information-sources)
+- [Installation](#installation)
+  - [Install the openstef package](#install-the-openstef-package)
+    - [Remark regarding installation within a **conda environment on Windows**:](#remark-regarding-installation-within-a-conda-environment-on-windows)
+- [Usage](#usage)
+  - [Reference Implementation](#reference-implementation)
+  - [Openstef-dbc - Database connector for openstef](#openstef-dbc---database-connector-for-openstef)
+  - [Example notebooks](#example-notebooks)
+  - [License](#license)
+  - [Licenses third-party libraries](#licenses-third-party-libraries)
+  - [Contributing](#contributing)
+  - [Contact](#contact)
+
+# External information sources
+- [Documentation website](https://openstef.github.io/openstef/index.html);
+- [Python package](https://pypi.org/project/openstef/);
+- [Project website](https://www.lfenergy.org/projects/openstef/);
+- [Documentation on dashboard](https://raw.githack.com/OpenSTEF/.github/main/profile/html/openstef_dashboard_doc.html);
+- [Linux Foundation project page](https://openstef.github.io/openstef/index.html)
+- [Video about OpenSTEF](https://www.lfenergy.org/forecasting-to-create-a-more-resilient-optimized-grid/);
+- [Teams channel](https://teams.microsoft.com/l/team/19%3ac08a513650524fc988afb296cd0358cc%40thread.tacv2/conversations?groupId=bfcb763a-3a97-4938-81d7-b14512aa537d&tenantId=697f104b-d7cb-48c8-ac9f-bd87105bafdc) 
 
 # Installation
 
@@ -53,10 +94,13 @@ OpenSTEF is a Python package which is used to make short term forecasts for the
 pip install openstef
 ```
 
-_**Optional**_: if you would like to use the proloaf model with OpenSTEF install the proloaf dependencies by running:
+### Remark regarding installation within a **conda environment on Windows**:
+
+A version of the pywin32 package will be installed as a secondary dependency along with the installation of the openstef package. Since conda relies on an old version of pywin32, the new installation can break conda's functionality. The following command can solve this issue:
 ```shell
-pip install openstef[proloaf]
+pip install pywin32==300
 ```
+For more information on this issue see the [readme of pywin32](https://github.com/mhammond/pywin32#installing-via-pip) or [this Github issue](https://github.com/mhammond/pywin32/issues/1865#issue-1212752696).
 
 # Usage
 
@@ -70,9 +114,15 @@ python -m openstef task <task_name>
 A complete implementation including databases, user interface, example data, etc. is available at: https://github.com/OpenSTEF/openstef-reference
 
 ![screenshot](https://user-images.githubusercontent.com/60883372/146760483-29af3ac7-62af-4f13-98c7-982a79c517d1.jpg)
-Screenshot of the operational dashboard showing the key functionality of OpenSTEF. 
+Screenshot of the operational dashboard showing the key functionality of OpenSTEF.
 Dashboard documentation can be found [here](https://github.com/OpenSTEF/.github/blob/main/profile/README.md).
 
+## Openstef-dbc - Database connector for openstef
+This repository provides an interface to OpenSTEF (reference) databases. The repository can be found [here](https://github.com/OpenSTEF/openstef-dbc).
+
+## Example notebooks 
+To help you get started, a set of fundamental example notebooks has been created. You can access these offline examples [here](https://github.com/OpenSTEF/openstef-offline-example).
+
 ## License
 This project is licensed under the Mozilla Public License, version 2.0 - see LICENSE for details.
 
@@ -80,9 +130,7 @@ This project is licensed under the Mozilla Public License, version 2.0 - see LIC
 This project includes third-party libraries, which are licensed under their own respective Open-Source licenses. SPDX-License-Identifier headers are used to show which license is applicable. The concerning license files can be found in the LICENSES directory.
 
 ## Contributing
-Please read [CODE_OF_CONDUCT.md](https://github.com/OpenSTEF/.github/blob/main/CODE_OF_CONDUCT.md), [CONTRIBUTING.md](https://github.com/OpenSTEF/.github/blob/main/CONTRIBUTING.md) and [PROJECT_GOVERNANACE.md](https://github.com/OpenSTEF/.github/blob/main/PROJECT_GOVERNANCE.md) for details on the process for submitting pull requests to us.
+Please read [CODE_OF_CONDUCT.md](https://github.com/OpenSTEF/.github/blob/main/CODE_OF_CONDUCT.md), [CONTRIBUTING.md](https://github.com/OpenSTEF/.github/blob/main/CONTRIBUTING.md) and [PROJECT_GOVERNANCE.md](https://github.com/OpenSTEF/.github/blob/main/PROJECT_GOVERNANCE.md) for details on the process for submitting pull requests to us.
 
 ## Contact
 Please read [SUPPORT.md](https://github.com/OpenSTEF/.github/blob/main/SUPPORT.md) for how to connect and get into contact with the OpenSTEF project
-
-

openstef-3.2.16/PKG-INFO → openstef-3.4.16/README.md

@@ -1,27 +1,5 @@
-Metadata-Version: 2.1
-Name: openstef
-Version: 3.2.16
-Summary: Open short term energy forecaster
-Home-page: https://github.com/OpenSTEF/openstef
-Author: Alliander N.V
-Author-email: korte.termijn.prognoses@alliander.com
-License: MPL-2.0
-Keywords: energy,forecasting,machinelearning
-Platform: UNKNOWN
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Intended Audience :: Developers
-Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
-Classifier: Programming Language :: Python :: 3.7
-Classifier: Programming Language :: Python :: 3.8
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Requires-Python: >=3.7.0
-Description-Content-Type: text/markdown
-Provides-Extra: proloaf
-License-File: LICENSE
-
 <!--
-SPDX-FileCopyrightText: 2017-2022 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
 
 SPDX-License-Identifier: MPL-2.0
 -->
@@ -40,10 +18,37 @@ SPDX-License-Identifier: MPL-2.0
 [![Technical Debt](https://sonarcloud.io/api/project_badges/measure?project=OpenSTEF_openstef&metric=sqale_index)](https://sonarcloud.io/dashboard?id=OpenSTEF_openstef)
 [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=OpenSTEF_openstef&metric=vulnerabilities)](https://sonarcloud.io/dashboard?id=OpenSTEF_openstef)
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5585/badge)](https://bestpractices.coreinfrastructure.org/projects/5585)
+[![Downloads](https://static.pepy.tech/badge/openstef)](https://pepy.tech/project/openstef)
+[![Downloads](https://static.pepy.tech/badge/openstef/month)](https://pepy.tech/project/openstef)
 
 # OpenSTEF
 
-OpenSTEF is a Python package which is used to make short term forecasts for the energy sector. This repository contains all components for the machine learning pipeline required to make a forecast. In order to use the package you need to provide your own data storage and retrieval interface. `openstef` is available at: https://pypi.org/project/openstef/
+OpenSTEF is a Python package designed for generating short-term forecasts in the energy sector. The repository includes all the essential components required for machine learning pipelines that facilitate the forecasting process. To utilize the package, users are required to furnish their own data storage and retrieval interface.
+
+# Table of contents
+- [OpenSTEF](#openstef)
+- [Table of contents](#table-of-contents)
+- [External information sources](#external-information-sources)
+- [Installation](#installation)
+  - [Install the openstef package](#install-the-openstef-package)
+    - [Remark regarding installation within a **conda environment on Windows**:](#remark-regarding-installation-within-a-conda-environment-on-windows)
+- [Usage](#usage)
+  - [Reference Implementation](#reference-implementation)
+  - [Openstef-dbc - Database connector for openstef](#openstef-dbc---database-connector-for-openstef)
+  - [Example notebooks](#example-notebooks)
+  - [License](#license)
+  - [Licenses third-party libraries](#licenses-third-party-libraries)
+  - [Contributing](#contributing)
+  - [Contact](#contact)
+
+# External information sources
+- [Documentation website](https://openstef.github.io/openstef/index.html);
+- [Python package](https://pypi.org/project/openstef/);
+- [Project website](https://www.lfenergy.org/projects/openstef/);
+- [Documentation on dashboard](https://raw.githack.com/OpenSTEF/.github/main/profile/html/openstef_dashboard_doc.html);
+- [Linux Foundation project page](https://openstef.github.io/openstef/index.html)
+- [Video about OpenSTEF](https://www.lfenergy.org/forecasting-to-create-a-more-resilient-optimized-grid/);
+- [Teams channel](https://teams.microsoft.com/l/team/19%3ac08a513650524fc988afb296cd0358cc%40thread.tacv2/conversations?groupId=bfcb763a-3a97-4938-81d7-b14512aa537d&tenantId=697f104b-d7cb-48c8-ac9f-bd87105bafdc) 
 
 # Installation
 
@@ -53,10 +58,13 @@ OpenSTEF is a Python package which is used to make short term forecasts for the
 pip install openstef
 ```
 
-_**Optional**_: if you would like to use the proloaf model with OpenSTEF install the proloaf dependencies by running:
+### Remark regarding installation within a **conda environment on Windows**:
+
+A version of the pywin32 package will be installed as a secondary dependency along with the installation of the openstef package. Since conda relies on an old version of pywin32, the new installation can break conda's functionality. The following command can solve this issue:
 ```shell
-pip install openstef[proloaf]
+pip install pywin32==300
 ```
+For more information on this issue see the [readme of pywin32](https://github.com/mhammond/pywin32#installing-via-pip) or [this Github issue](https://github.com/mhammond/pywin32/issues/1865#issue-1212752696).
 
 # Usage
 
@@ -70,9 +78,15 @@ python -m openstef task <task_name>
 A complete implementation including databases, user interface, example data, etc. is available at: https://github.com/OpenSTEF/openstef-reference
 
 ![screenshot](https://user-images.githubusercontent.com/60883372/146760483-29af3ac7-62af-4f13-98c7-982a79c517d1.jpg)
-Screenshot of the operational dashboard showing the key functionality of OpenSTEF. 
+Screenshot of the operational dashboard showing the key functionality of OpenSTEF.
 Dashboard documentation can be found [here](https://github.com/OpenSTEF/.github/blob/main/profile/README.md).
 
+## Openstef-dbc - Database connector for openstef
+This repository provides an interface to OpenSTEF (reference) databases. The repository can be found [here](https://github.com/OpenSTEF/openstef-dbc).
+
+## Example notebooks 
+To help you get started, a set of fundamental example notebooks has been created. You can access these offline examples [here](https://github.com/OpenSTEF/openstef-offline-example).
+
 ## License
 This project is licensed under the Mozilla Public License, version 2.0 - see LICENSE for details.
 
@@ -80,9 +94,7 @@ This project is licensed under the Mozilla Public License, version 2.0 - see LIC
 This project includes third-party libraries, which are licensed under their own respective Open-Source licenses. SPDX-License-Identifier headers are used to show which license is applicable. The concerning license files can be found in the LICENSES directory.
 
 ## Contributing
-Please read [CODE_OF_CONDUCT.md](https://github.com/OpenSTEF/.github/blob/main/CODE_OF_CONDUCT.md), [CONTRIBUTING.md](https://github.com/OpenSTEF/.github/blob/main/CONTRIBUTING.md) and [PROJECT_GOVERNANACE.md](https://github.com/OpenSTEF/.github/blob/main/PROJECT_GOVERNANCE.md) for details on the process for submitting pull requests to us.
+Please read [CODE_OF_CONDUCT.md](https://github.com/OpenSTEF/.github/blob/main/CODE_OF_CONDUCT.md), [CONTRIBUTING.md](https://github.com/OpenSTEF/.github/blob/main/CONTRIBUTING.md) and [PROJECT_GOVERNANCE.md](https://github.com/OpenSTEF/.github/blob/main/PROJECT_GOVERNANCE.md) for details on the process for submitting pull requests to us.
 
 ## Contact
 Please read [SUPPORT.md](https://github.com/OpenSTEF/.github/blob/main/SUPPORT.md) for how to connect and get into contact with the OpenSTEF project
-
-

{openstef-3.2.16 → openstef-3.4.16}/openstef/__init__.py

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: 2017-2022 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
 #
 # SPDX-License-Identifier: MPL-2.0
 

openstef-3.4.16/openstef/__main__.py

@@ -0,0 +1,3 @@
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
+#
+# SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_adaptation_model.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_adaptation_model_features.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_adaptation_model_scaler.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_domain_model.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_domain_model_features.z

@@ -0,0 +1,2 @@
+x^%�;
+�0DS����#U�}� $c��

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_domain_model_features.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_domain_model_scaler.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_model_card.md

@@ -0,0 +1,14 @@
+"
+    # Model details : dazls_model_test_eval
+    
+    ## Description
+    **Model Name**: dazls_model_test_eval
+    **Author**: KTP, Alliander
+    **Model type**: Energy splitting model
+    **Model Architecture**: KNeighestNeighbours
+    **Date**: 2024-01-31
+    
+    ## Intended use
+    This is a DAZLs model aimed at determining the energy splits for substations. 
+    Each of these splits are determined based on a set of features that are available in production,
+    and in this case have their origin in the Dutch energy grid. 

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_model_card.md.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_target.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dazls_model_3.4.7/dazls_stored_3.4.7_target_scaler.z.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/dutch_holidays_2020-2022.csv.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data/pv_single_coefs.csv.license

@@ -0,0 +1,3 @@
+SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+
+SPDX-License-Identifier: MPL-2.0

{openstef-3.2.16/openstef/model → openstef-3.4.16/openstef/data_classes}/__init__.py

@@ -1,3 +1,3 @@
-# SPDX-FileCopyrightText: 2017-2022 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
 #
 # SPDX-License-Identifier: MPL-2.0

openstef-3.4.16/openstef/data_classes/data_prep.py

@@ -0,0 +1,99 @@
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
+#
+# SPDX-License-Identifier: MPL-2.0
+"""Specifies the split function dataclass."""
+import inspect
+import json
+from importlib import import_module
+from typing import Any, Sequence, TypeVar, Union
+
+from pydantic.v1 import BaseModel
+
+DataPrepClass = TypeVar("DataPrepClass")
+
+
+class DataPrepDataClass(BaseModel):
+    """Class that allows to specify a custom class to prepare the data (feature engineering , etc ...)."""
+
+    klass: Union[str, type[DataPrepClass]]
+    arguments: Union[
+        str, dict[str, Any]
+    ]  # JSON string holding the function parameters or dict
+
+    def __getitem__(self, key: str):
+        """Allows us to use subscription to get the items from the object."""
+        return getattr(self, key)
+
+    def __setitem__(self, key: str, value: any):
+        """Allows us to use subscription to set the items in the object."""
+        if hasattr(self, key):
+            self.__dict__[key] = value
+        else:
+            raise AttributeError(f"{key} not an attribute of prediction job.")
+
+    def _load_klass(
+        self, required_arguments: Sequence[str] = None
+    ) -> type[DataPrepClass]:
+        """Load data prep class from path.
+
+        Args:
+            klass_path (str): The path to the data prep class
+
+        Returns:
+            klass (type[AbstractDataPreparation]): The loaded data prep class
+
+        """
+        if isinstance(self.klass, str):
+            path_elements = self.klass.split(".")
+            module_path = ".".join(path_elements[:-1])
+            module = import_module(module_path)
+            klass_name = path_elements[-1]
+            klass = getattr(module, klass_name)
+        else:
+            klass = self.klass
+
+        # Check that the klass accepts mandatory arguments
+        if not inspect.isclass(klass):
+            raise ValueError("The loaded object is not a class: {klass!r}")
+
+        if required_arguments is not None:
+            klass_params = set(inspect.signature(klass).parameters)
+
+            if len(set(required_arguments) - klass_params) > 0:
+                raise ValueError(
+                    "The loaded data prep class does not have the required arguments"
+                )
+
+        return klass
+
+    def _load_arguments(self) -> dict[str, Any]:
+        """Load the arguments.
+
+        Convert the arguments from JSON if they are given as strings or simply return them otherwise.
+
+        Returns:
+            arguments (dict[str, Any]): The additional arguments to be passed to the class
+
+        """
+        if isinstance(self.arguments, str):
+            return json.loads(self.arguments)
+        else:
+            return self.arguments
+
+    def load(
+        self, required_arguments: Sequence[str] = None
+    ) -> tuple[type[DataPrepClass], dict[str, Any]]:
+        """Load the function and its arguments.
+
+        If the function and the arguments are given as strings in the instane attributes, load them as Python objects
+        otherwise just return them from the instance attributes.
+
+        Args:
+            required_arguments (list[str]): list of arguments the loaded class must have
+
+        Returns:
+            - class (type[AbstractDataPreparation])
+            - arguments (dict[str, Any])
+
+        """
+        return self._load_klass(required_arguments), self._load_arguments()

{openstef-3.2.16 → openstef-3.4.16}/openstef/data_classes/model_specifications.py

@@ -1,24 +1,29 @@
-# SPDX-FileCopyrightText: 2017-2022 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com>
 #
 # SPDX-License-Identifier: MPL-2.0
-
+"""Specifies the dataclass for model specifications."""
 from typing import Optional, Union
 
-from pydantic import BaseModel
+from pydantic.v1 import BaseModel
 
 
 class ModelSpecificationDataClass(BaseModel):
+    """Holds all information regarding the training procces of a specific model."""
+
     id: Union[int, str]
     hyper_params: Optional[dict] = {}
+    """Hyperparameters that should be used during training."""
     feature_names: Optional[list] = None
+    """Features that should be used during training."""
     feature_modules: Optional[list] = []
+    """Feature modules that should be used during training."""
 
-    def __getitem__(self, item):
-        """Allows us to use subscription to get the items from the object"""
+    def __getitem__(self, item: str) -> any:
+        """Allows us to use subscription to get the items from the object."""
         return getattr(self, item)
 
-    def __setitem__(self, key: str, value: any):
-        """Allows us to use subscription to set the items in the object"""
+    def __setitem__(self, key: str, value: any) -> None:
+        """Allows us to use subscription to set the items in the object."""
         if hasattr(self, key):
             self.__dict__[key] = value
         else:

openstef-3.4.16/openstef/data_classes/prediction_job.py

@@ -0,0 +1,122 @@
+# SPDX-FileCopyrightText: 2017-2023 Contributors to the OpenSTEF project <korte.termijn.prognoses@alliander.com> # noqa E501>
+#
+# SPDX-License-Identifier: MPL-2.0
+"""Specifies the prediction job dataclass."""
+from typing import Optional, Union
+
+from pydantic.v1 import BaseModel
+
+from openstef.data_classes.data_prep import DataPrepDataClass
+from openstef.data_classes.model_specifications import ModelSpecificationDataClass
+from openstef.data_classes.split_function import SplitFuncDataClass
+from openstef.enums import PipelineType
+
+
+class PredictionJobDataClass(BaseModel):
+    """Holds all information about the specific forecast that has to be made."""
+
+    id: Union[int, str]
+    """The predictions job id (often abreviated as pid)."""
+    model: str
+    """The model type that should be used.
+
+    Options are:
+        - ``"xgb"``
+        - ``"xgb_quantile"``
+        - ``"lgb"``
+        - ``"linear"``
+
+    If unsure what to pick, choose ``"xgb"``.
+
+    """
+    forecast_type: str
+    """The type of forecasts that should be made.
+
+    Options are:
+        - ``"demand"``
+        - ``"wind"``
+        - ``"basecase"``
+
+    If unsure what to pick, choose ``"demand"``.
+
+    """
+    horizon_minutes: Optional[int] = 2880
+    """The horizon of the desired forecast in minutes used in tasks. Defaults to 2880 minutes (i.e. 2 days)."""
+    resolution_minutes: int
+    """The resolution of the desired forecast in minutes."""
+    lat: Optional[float] = 52.132633
+    """Latitude of the forecasted location in degrees. Used for fetching weather data in tasks, calculating derrived features and component splitting."""
+    lon: Optional[float] = 5.291266
+    """Longitude of the forecasted location in degrees. Used for fetching weather data in tasks, calculating derrived features and component splitting."""
+    name: str
+    """Name of the forecast, e.g. the location name."""
+    train_components: Optional[bool]
+    """Whether splitting the forecasts in wind, solar, rest is desired."""
+    description: Optional[str]
+    """Optional description of the prediction job for human reference."""
+    quantiles: Optional[list[float]]
+    """Quantiles that have to be forecasted."""
+    train_split_func: Optional[SplitFuncDataClass]
+    """Optional custom splitting function for operational procces."""
+    backtest_split_func: Optional[SplitFuncDataClass]
+    """Optional custom splitting function for backtesting."""
+    train_horizons_minutes: Optional[list[int]]
+    """List of horizons that should be taken into account during training."""
+    default_modelspecs: Optional[ModelSpecificationDataClass]
+    """Default model specifications"""
+    save_train_forecasts: bool = False
+    """Indicate wether the forecasts produced during the training process should be saved."""
+    completeness_threshold: float = 0.5
+    """Minimum fraction of data that should be available for making a regular forecast."""
+    minimal_table_length: int = 100
+    """Minimum length (in rows) of the forecast input for making a regular forecast."""
+    flatliner_threshold_minutes: int = 1440
+    """Number of minutes that the load has to be constant to detect a flatliner. """
+    depends_on: Optional[list[Union[int, str]]]
+    """Link to another prediction job on which this prediction job might depend."""
+    sid: Optional[str]
+    """Only required for create_solar_forecast task"""
+    turbine_type: Optional[str]
+    """Only required for create_wind_forecast task"""
+    n_turbines: Optional[float]
+    """Only required for create_wind_forecast task"""
+    hub_height: Optional[float]
+    """Only required for create_wind_forecast task"""
+    pipelines_to_run: list[PipelineType] = [
+        PipelineType.TRAIN,
+        PipelineType.HYPER_PARMATERS,
+        PipelineType.FORECAST,
+    ]
+    """The pipelines to run for this pj"""
+    alternative_forecast_model_pid: Optional[Union[int, str]]
+    """The pid that references another prediction job from which the model should be used for making forecasts."""
+    data_prep_class: Optional[DataPrepDataClass]
+    """The import string for the custom data prep class"""
+
+    class Config:
+        """Pydantic model configuration.
+
+        This following configuration is needed to prevent ids in "depends_on" to be converted from int to str when we
+        use integer ids.
+
+        """
+
+        smart_union = True
+
+    def __getitem__(self, item: str) -> any:
+        """Allows us to use subscription to get the items from the object."""
+        return getattr(self, item)
+
+    def __setitem__(self, key: str, value: any) -> None:
+        """Allows us to use subscription to set the items in the object."""
+        if hasattr(self, key):
+            self.__dict__[key] = value
+        else:
+            raise AttributeError(f"{key} not an attribute of prediction job.")
+
+    def get(self, key: str, default: any = None) -> any:
+        """Allows to use the get functions similar to a python dict."""
+        if hasattr(self, key):
+            return getattr(self, key)
+        else:
+            return default