views_pipeline_core 0.1.0__tar.gz

views_pipeline_core-0.1.0/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 views platform
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

views_pipeline_core-0.1.0/PKG-INFO

@@ -0,0 +1,96 @@
+Metadata-Version: 2.1
+Name: views_pipeline_core
+Version: 0.1.0
+Summary: 
+Author: Borbála Farkas
+Author-email: borbala.farkas@pcr.uu.se
+Requires-Python: >=3.11,<3.15
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: ingester3 (>=2.1.0,<3.0.0)
+Requires-Dist: properscoring (>=0.1,<0.2)
+Requires-Dist: pytest (>=8.3.3,<9.0.0)
+Requires-Dist: views_forecasts (>=0.5.5,<0.6.0)
+Requires-Dist: viewser (>=6.6.4,<7.0.0)
+Requires-Dist: wandb (>=0.18.7,<0.19.0)
+Description-Content-Type: text/markdown
+
+# views_pipeline_core
+
+<div style="width: 100%; max-width: 1500px; height: 400px; overflow: hidden; position: relative;">
+  <img src="https://pbs.twimg.com/profile_banners/1237000633896652800/1717069203/1500x500" alt="VIEWS Twitter Header" style="position: absolute; top: -50px; width: 100%; height: auto;">
+</div>
+
+The [**Violence & Impacts Early Warning System (VIEWS)**](https://viewsforecasting.org/) produces monthly predictions of future violent conflict at both a country and sub-country level of analysis. This repository contains code, configuration files, and documentation that encapsulates the entire process of developing, experimenting, training, evaluating, and deploying the VIEWS machine learning model pipeline.
+
+Use our [interactive data dashboard](https://data.viewsforecasting.org/) to explore our latest predictions of future armed conflict.
+
+> [!CAUTION]
+> Please note that this pipeline is **actively under construction**. We're in the **early stages of development**, meaning it's **not yet ready for operational use**. We're working hard to bring you a robust and fully-functional tool, so stay tuned for updates!
+
+## Table of contents
+
+<!-- toc -->
+- [Repository Contents, Structure, and Explanations](#repository-contents-structure-and-explanations)
+- [Pipeline Execution](#pipeline-execution)
+- [Pipeline Documentation](#pipeline-documentation)
+- [About the VIEWS Project](#about-the-views-project)
+
+
+<!-- tocstop -->
+
+
+## Repository Contents, Structure, and Explanations
+![VIEWS pipeline diagram](documentation/pipeline_diagram001.png)
+
+### Repository Contents
+
+This repository includes:
+
+- **Code:** Source code for the VIEWS project's machine learning models and the full pipeline.
+- **Configuration Files:** Settings and configurations for running the models, ensembles, and orchestration scripts.
+- **Documentation:** Detailed instructions and information about the project and how to interact with the pipeline and the individual components.
+
+### Pipeline Overview
+
+The VIEWS machine learning pipeline involves several key processes:
+
+- **Developing:** Creating and refining machine learning models.
+- **Experimentation:** Testing and validating various model configurations and approaches.
+- **Training:** Training models with relevant data.
+- **Evaluating:** Assessing model performance and accuracy.
+- **Deploying:** Implementing models in a production environment to generate monthly true-future forecasts 
+
+## Pipeline Documentation
+High-level documentation on the pipeline and its components can be found in the folder [`documentation`](https://github.com/views-platform/views_pipeline_core/tree/main/documentation). For a comprehensive understanding of the terms and concepts used, please consult the [`Glossary`](https://github.com/views-platform/views_pipeline_core/blob/main/documentation/glossary.md). To explore the rationale behind our architectural choices, visit the [`Architectural Decision Records (ADRs)`](https://github.com/views-platform/views_pipeline_core/blob/main/documentation/ADRs/README.md).
+
+Additionally, refer to READMEs and docstrings of various functions and classes in the source code.
+
+The operational fatalities model generates forecasts for state-based armed conflict during each month in a rolling 3-year window. 
+The latest iteration, currently in production, is called [Fatalities002](https://viewsforecasting.org/early-warning-system/models/fatalities002/).
+
+The following links cover **modelling documentation** for Fatalities002:
+- [Prediction models and input variables in main ensemble](https://viewsforecasting.org/views_documentation_models_fatalities002/)
+- [Levels of analysis and dependent variables](https://viewsforecasting.org/wp-content/uploads/VIEWS_documentation_LevelsandOutcomes.pdf)
+- [Partitioning and time shifting data for training, calibration, testing/forecasting, model weighting, and out-of-sample evaluation](https://viewsforecasting.org/wp-content/uploads/VIEWS_Documentation_Partitioningandtimeshifting_Fatalities002.pdf)
+- [Ensembling and calibration](https://viewsforecasting.org/wp-content/uploads/VIEWS_documentation_Ensembling_Fatalities002.pdf)
+
+For VIEWS-specific **infrastructure documentation**, please refer to following GitHub repositories:
+- [`ingester3`: Loading input data into the views database](https://github.com/UppsalaConflictDataProgram/ingester3)
+- [`viewser`: Accessing input data from views database](https://github.com/prio-data/viewser)
+- [`views_api`: Our API for accessing predictions](https://github.com/prio-data/views_api)
+
+## About the VIEWS Project
+
+The VIEWS project is a collaborative effort supported by leading research institutions focused on peace and conflict studies. For more information about the project, visit the [VIEWS Forecasting webpage](https://viewsforecasting.org/).
+
+### Affiliations
+
+- **Peace Research Institute Oslo (PRIO):**
+  The [Peace Research Institute Oslo (PRIO)](https://www.prio.org/) conducts research on the conditions for peaceful relations between states, groups, and people. PRIO is dedicated to understanding the processes that lead to violence and those that create sustainable peace. About half of the VIEWS core team is currently located at PRIO.
+
+- **Department of Peace and Conflict Research at the University of Uppsala:**
+  The [Department of Peace and Conflict Research at the University of Uppsala](https://www.uu.se/en/department/peace-and-conflict-research) is a leading academic institution in the study of conflict resolution, peacebuilding, and security. The department is renowned for its research and education programs aimed at fostering a deeper understanding of conflict dynamics and peace processes. This department also hosts the [Uppsala Conflict Data Program (UCDP)](https://ucdp.uu.se/), a central data source for the VIEWS project. About half of the VIEWS core team is currently located at the University of Uppsala.
+

views_pipeline_core-0.1.0/README.md

@@ -0,0 +1,76 @@
+# views_pipeline_core
+
+<div style="width: 100%; max-width: 1500px; height: 400px; overflow: hidden; position: relative;">
+  <img src="https://pbs.twimg.com/profile_banners/1237000633896652800/1717069203/1500x500" alt="VIEWS Twitter Header" style="position: absolute; top: -50px; width: 100%; height: auto;">
+</div>
+
+The [**Violence & Impacts Early Warning System (VIEWS)**](https://viewsforecasting.org/) produces monthly predictions of future violent conflict at both a country and sub-country level of analysis. This repository contains code, configuration files, and documentation that encapsulates the entire process of developing, experimenting, training, evaluating, and deploying the VIEWS machine learning model pipeline.
+
+Use our [interactive data dashboard](https://data.viewsforecasting.org/) to explore our latest predictions of future armed conflict.
+
+> [!CAUTION]
+> Please note that this pipeline is **actively under construction**. We're in the **early stages of development**, meaning it's **not yet ready for operational use**. We're working hard to bring you a robust and fully-functional tool, so stay tuned for updates!
+
+## Table of contents
+
+<!-- toc -->
+- [Repository Contents, Structure, and Explanations](#repository-contents-structure-and-explanations)
+- [Pipeline Execution](#pipeline-execution)
+- [Pipeline Documentation](#pipeline-documentation)
+- [About the VIEWS Project](#about-the-views-project)
+
+
+<!-- tocstop -->
+
+
+## Repository Contents, Structure, and Explanations
+![VIEWS pipeline diagram](documentation/pipeline_diagram001.png)
+
+### Repository Contents
+
+This repository includes:
+
+- **Code:** Source code for the VIEWS project's machine learning models and the full pipeline.
+- **Configuration Files:** Settings and configurations for running the models, ensembles, and orchestration scripts.
+- **Documentation:** Detailed instructions and information about the project and how to interact with the pipeline and the individual components.
+
+### Pipeline Overview
+
+The VIEWS machine learning pipeline involves several key processes:
+
+- **Developing:** Creating and refining machine learning models.
+- **Experimentation:** Testing and validating various model configurations and approaches.
+- **Training:** Training models with relevant data.
+- **Evaluating:** Assessing model performance and accuracy.
+- **Deploying:** Implementing models in a production environment to generate monthly true-future forecasts 
+
+## Pipeline Documentation
+High-level documentation on the pipeline and its components can be found in the folder [`documentation`](https://github.com/views-platform/views_pipeline_core/tree/main/documentation). For a comprehensive understanding of the terms and concepts used, please consult the [`Glossary`](https://github.com/views-platform/views_pipeline_core/blob/main/documentation/glossary.md). To explore the rationale behind our architectural choices, visit the [`Architectural Decision Records (ADRs)`](https://github.com/views-platform/views_pipeline_core/blob/main/documentation/ADRs/README.md).
+
+Additionally, refer to READMEs and docstrings of various functions and classes in the source code.
+
+The operational fatalities model generates forecasts for state-based armed conflict during each month in a rolling 3-year window. 
+The latest iteration, currently in production, is called [Fatalities002](https://viewsforecasting.org/early-warning-system/models/fatalities002/).
+
+The following links cover **modelling documentation** for Fatalities002:
+- [Prediction models and input variables in main ensemble](https://viewsforecasting.org/views_documentation_models_fatalities002/)
+- [Levels of analysis and dependent variables](https://viewsforecasting.org/wp-content/uploads/VIEWS_documentation_LevelsandOutcomes.pdf)
+- [Partitioning and time shifting data for training, calibration, testing/forecasting, model weighting, and out-of-sample evaluation](https://viewsforecasting.org/wp-content/uploads/VIEWS_Documentation_Partitioningandtimeshifting_Fatalities002.pdf)
+- [Ensembling and calibration](https://viewsforecasting.org/wp-content/uploads/VIEWS_documentation_Ensembling_Fatalities002.pdf)
+
+For VIEWS-specific **infrastructure documentation**, please refer to following GitHub repositories:
+- [`ingester3`: Loading input data into the views database](https://github.com/UppsalaConflictDataProgram/ingester3)
+- [`viewser`: Accessing input data from views database](https://github.com/prio-data/viewser)
+- [`views_api`: Our API for accessing predictions](https://github.com/prio-data/views_api)
+
+## About the VIEWS Project
+
+The VIEWS project is a collaborative effort supported by leading research institutions focused on peace and conflict studies. For more information about the project, visit the [VIEWS Forecasting webpage](https://viewsforecasting.org/).
+
+### Affiliations
+
+- **Peace Research Institute Oslo (PRIO):**
+  The [Peace Research Institute Oslo (PRIO)](https://www.prio.org/) conducts research on the conditions for peaceful relations between states, groups, and people. PRIO is dedicated to understanding the processes that lead to violence and those that create sustainable peace. About half of the VIEWS core team is currently located at PRIO.
+
+- **Department of Peace and Conflict Research at the University of Uppsala:**
+  The [Department of Peace and Conflict Research at the University of Uppsala](https://www.uu.se/en/department/peace-and-conflict-research) is a leading academic institution in the study of conflict resolution, peacebuilding, and security. The department is renowned for its research and education programs aimed at fostering a deeper understanding of conflict dynamics and peace processes. This department also hosts the [Uppsala Conflict Data Program (UCDP)](https://ucdp.uu.se/), a central data source for the VIEWS project. About half of the VIEWS core team is currently located at the University of Uppsala.

views_pipeline_core-0.1.0/pyproject.toml

@@ -0,0 +1,25 @@
+[tool.poetry]
+name = "views_pipeline_core"
+version = "0.1.0"
+description = ""
+authors = [
+    "Borbála Farkas <borbala.farkas@pcr.uu.se>",
+    "Dylan Pinheiro <dylpin@prio.org>",
+    "Xiaolong Sun <xiaolong.sun@pcr.uu.se>"
+]
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = ">=3.11,<3.15"
+viewser = "^6.6.4"
+pytest = "^8.3.3"
+ingester3 = "^2.1.0"
+views_forecasts = "^0.5.5"
+properscoring = "^0.1"
+wandb = "^0.18.7"
+
+
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

views_pipeline_core-0.1.0/views_pipeline_core/cache/global_cache.py

@@ -0,0 +1,237 @@
+import pickle
+import os
+import logging
+import sys
+from pathlib import Path
+import threading
+import atexit
+import signal
+from views_pipeline_core.managers.path_manager import ModelPath
+# Configure logging
+logging.basicConfig(
+    level=logging.DEBUG, format="%(asctime)s - %(levelname)s - %(message)s"
+)
+
+
+class GlobalCacheMeta(type):
+    def __getitem__(cls, key):
+        instance = cls()
+        return instance[key]
+
+    def __setitem__(cls, key, value):
+        instance = cls()
+        instance[key] = value
+
+    def delete(cls, key):
+        instance = cls()
+        instance._delete(key)
+
+class GlobalCache(metaclass=GlobalCacheMeta):
+    """
+    A thread-safe singleton cache class that uses a global cache file to store key-value pairs.
+
+    Attributes:
+        _instance (GlobalCache): The singleton instance of the GlobalCache class.
+        _lock (threading.Lock): A lock to ensure thread safety.
+        filepath (Path): The path to the cache file.
+        cache (dict): The in-memory cache dictionary.
+        initialized (bool): A flag to check if the instance is initialized.
+    """
+
+    _instance = None
+    _lock = threading.Lock()
+
+    def __new__(cls, *args, **kwargs):
+        """
+        Ensures that only one instance of the GlobalCache class is created (singleton pattern).
+
+        Uses double-checked locking to minimize the overhead of acquiring the lock.
+        """
+        if cls._instance is None:
+            with cls._lock:
+                if cls._instance is None:
+                    cls._instance = super(GlobalCache, cls).__new__(cls)
+        return cls._instance
+
+    def __init__(self, filepath=None):
+        """
+        Initializes the GlobalCache instance.
+
+        Args:
+            filepath (Path, optional): The path to the cache file. Defaults to '.global_cache.pkl' in the project root.
+        """
+        if not hasattr(self, "initialized"):
+            self.local_imports()
+            if filepath is None:
+                filepath = (
+                    ModelPath.find_project_root() / ".global_cache.pkl"
+                )
+            self.filepath = filepath
+            self.cache = {}
+            self.ensure_cache_file_exists()
+            self.load_cache()
+            self.initialized = True
+
+    def __getitem__(self, key):
+        """
+        Retrieves a value from the cache by key.
+
+        Args:
+            key (str): The key to retrieve the value for.
+
+        Returns:
+            The value associated with the key, or None if the key is not found.
+        """
+        with self._lock:
+            return self.get(key)
+
+    def __setitem__(self, key, value):
+        """
+        Sets a value in the cache for a given key.
+
+        Args:
+            key (str): The key to set the value for.
+            value: The value to set.
+        """
+        with self._lock:
+            self.set(key, value)
+
+    def local_imports(self):
+        pass
+
+    def ensure_cache_file_exists(self):
+        """
+        Ensures that the cache file exists. If it does not exist, creates a new cache file.
+        """
+        if not self.filepath.exists():
+            logging.warning(
+                f"Cache file: {self.filepath} does not exist. Creating new cache file..."
+            )
+            with open(self.filepath, "wb") as f:
+                pickle.dump({}, f)
+            logging.debug(f"Created new cache file: {self.filepath}")
+
+    def set(self, key, value):
+        """
+        Sets a value in the in-memory cache and saves the cache to the file.
+
+        Args:
+            key (str): The key to set the value for.
+            value: The value to set.
+        """
+        self.cache[key] = value
+        self.save_cache()
+        logging.debug(f"Set key: {key} with value: {value}")
+
+    def get(self, key):
+        """
+        Retrieves a value from the in-memory cache by key.
+
+        Args:
+            key (str): The key to retrieve the value for.
+
+        Returns:
+            The value associated with the key, or None if the key is not found.
+        """
+        self.load_cache()
+        value = self.cache.get(key, None)
+        logging.debug(f"Get key: {key} returned value: {value}")
+        return value
+
+    def _delete(self, key):
+        """
+        Deletes a key-value pair from the in-memory cache and saves the cache to the file.
+        
+        Args:
+            key (str): The key to delete.
+        """
+        if key in self.cache:
+            del self.cache[key]
+            self.save_cache()
+            logging.debug(f'Deleted key: {key}')
+        else:
+            logging.warning(f'Key: {key} not found in cache')
+
+    def save_cache(self):
+        """
+        Saves the in-memory cache to the cache file.
+        """
+        with open(self.filepath, "wb") as f:
+            pickle.dump(self.cache, f)
+        logging.debug(f"Cache saved to file: {self.filepath}")
+
+    def load_cache(self):
+        """
+        Loads the cache from the cache file into the in-memory cache.
+        """
+        if self.filepath.exists():
+            try:
+                with open(str(self.filepath), "rb") as f:
+                    loaded_cache = pickle.loads(f.read())
+                    if isinstance(loaded_cache, dict):
+                        self.cache = loaded_cache
+                        logging.debug(f"Cache loaded from file: {self.filepath}")
+                    else:
+                        logging.error(
+                            f"Loaded cache is not a dictionary. Initializing empty cache."
+                        )
+                        self.cache = {}
+            except (EOFError, pickle.UnpicklingError) as e:
+                logging.error(
+                    f"Failed to load cache from file: {self.filepath}. Error: {e}"
+                )
+                self.cache = {}
+        else:
+            self.cache = {}
+            logging.debug(f"Cache file does not exist. Initialized empty cache.")
+
+
+def cleanup_cache_file():
+    """
+    Deletes the global cache file upon program termination.
+
+    This function is registered with the atexit module to ensure that the
+    global cache file is deleted when the program exits normally. It acquires
+    the lock to ensure thread safety and then checks if the cache file exists.
+    If the file exists, it deletes the file and logs the action.
+    """
+    try:
+        # Acquire the lock to ensure thread safety
+        with GlobalCache._lock:
+            if os.path.exists(GlobalCache().filepath):
+                os.remove(GlobalCache().filepath)
+                logging.info(f"Cache file: {GlobalCache().filepath} deleted at exit")
+    except Exception as e:
+        logging.error(f"Error during cleanup: {e}")
+
+
+def signal_handler(sig, frame):
+    """
+    Signal handler for SIGINT to delete the cache file upon user interruption.
+
+    Args:
+        sig (int): The signal number.
+        frame (FrameType): The current stack frame.
+    """
+    logging.info("SIGINT received. Deleting cache file...")
+    cleanup_cache_file()
+    os._exit(0)  # Use os._exit to ensure the process exits immediately
+
+
+# Register the cleanup_cache_file function to be called upon normal program termination
+atexit.register(cleanup_cache_file)
+
+# Register the signal handler for SIGINT
+signal.signal(signal.SIGINT, signal_handler)
+
+
+# if __name__ == '__main__':
+#     # Example usage of the GlobalCache class
+#     print(GlobalCache["test"]) # Returns None
+
+#     GlobalCache["test"] = "Hello, World!"
+#     print(GlobalCache["test"])
+#     GlobalCache.delete("test")
+
+#     print(GlobalCache["test"]) # Returns None
+

views_pipeline_core-0.1.0/views_pipeline_core/cli/utils.py

@@ -0,0 +1,143 @@
+import sys
+import argparse
+
+
+def parse_args():
+    """
+    CLI parser for model specific main.py scripts.
+    """
+
+    parser = argparse.ArgumentParser(
+        description="Run model pipeline with specified run type."
+    )
+
+    parser.add_argument(
+        "-r",
+        "--run_type",
+        choices=["calibration", "testing", "forecasting"],
+        type=str,
+        default="calibration",
+        help="Choose the run type for the model: calibration, testing, or forecasting. Default is calibration. "
+        "Note: If --sweep is flagged, --run_type must be calibration.",
+    )
+
+    parser.add_argument(
+        "-s",
+        "--sweep",
+        action="store_true",
+        help="Set flag to run the model pipeline as part of a sweep. No explicit flag means no sweep."
+        "Note: If --sweep is flagged, --run_type must be calibration, and both training and evaluation is automatically implied.",
+    )
+
+    parser.add_argument(
+        "-t",
+        "--train",
+        action="store_true",
+        help="Flag to indicate if a new model should be trained. "
+        "Note: If --sweep is flagged, --train will also automatically be flagged.",
+    )
+
+    parser.add_argument(
+        "-e",
+        "--evaluate",
+        action="store_true",
+        help="Flag to indicate if the model should be evaluated. "
+        "Note: If --sweep is specified, --evaluate will also automatically be flagged. "
+        "Cannot be used with --run_type forecasting.",
+    )
+
+    parser.add_argument(
+        "-f",
+        "--forecast",
+        action="store_true",
+        help="Flag to indicate if the model should produce predictions. "
+        "Note: If --sweep is specified, --forecast will also automatically be flagged. "
+        "Can only be used with --run_type forecasting.",
+    )
+
+    parser.add_argument(
+        "-a",
+        "--artifact_name",
+        type=str,
+        help="Specify the name of the model artifact to be used for evaluation. "
+        "The file extension will be added in the main and fit with the specific model algorithm."
+        "The artifact name should be in the format: <run_type>_model_<timestamp>.pt."
+        "where <run_type> is calibration, testing, or forecasting, and <timestamp> is in the format YMD_HMS."
+        "If not provided, the latest artifact will be used by default.",
+    )
+
+    parser.add_argument(
+        "-en",
+        "--ensemble",
+        action="store_true",
+        help="Flag to indicate if the model is an ensemble.",
+    )
+
+    parser.add_argument(
+        "-sa", "--saved", action="store_true", help="Used locally stored data"
+    )
+
+    parser.add_argument(
+        "-o", "--override_month", help="Over-ride use of current month", type=int
+    )
+
+    parser.add_argument(
+        "-dd", "--drift_self_test", action="store_true", default=False,
+        help="Enable drift-detection self_test at data-fetch"
+    )
+
+    return parser.parse_args()
+
+
+def validate_arguments(args):
+    if args.sweep and args.run_type != "calibration":
+        print("Error: Sweep runs must have --run_type set to 'calibration'. Exiting.")
+        print("To fix: Use --run_type calibration when --sweep is flagged.")
+        sys.exit(1)
+
+    if args.evaluate and args.run_type == "forecasting":
+        print("Error: Forecasting runs cannot evaluate. Exiting.")
+        print("To fix: Remove --evaluate flag when --run_type is 'forecasting'.")
+        sys.exit(1)
+
+    if (
+        args.run_type in ["calibration", "testing", "forecasting"]
+        and not args.train
+        and not args.evaluate
+        and not args.forecast
+        and not args.sweep
+    ):
+        print(
+            f"Error: Run type is {args.run_type} but neither --train, --evaluate, nor --sweep flag is set. Nothing to do... Exiting."
+        )
+        print(
+            "To fix: Add --train and/or --evaluate flag. Or use --sweep to run both training and evaluation in a WadnB sweep loop."
+        )
+        sys.exit(1)
+
+    if args.train and args.artifact_name:
+        print("Error: Both --train and --artifact_name flags are set. Exiting.")
+        print("To fix: Remove --artifact_name if --train is set, or vice versa.")
+        sys.exit(1)
+
+    if args.forecast and args.run_type != "forecasting":
+        print(
+            "Error: --forecast flag can only be used with --run_type forecasting. Exiting."
+        )
+        print("To fix: Set --run_type to forecasting if --forecast is flagged.")
+        sys.exit(1)
+
+    if args.ensemble and args.sweep:
+        # This is a temporary solution. In the future we might need to train and sweep the ensemble models.
+        print(
+            "Error: --aggregation flag cannot be used with --sweep. Exiting."
+        )
+        sys.exit(1)
+
+    if not args.train and not args.saved:
+        # if not training, then we need to use saved data
+        print(
+            "Error: if --train is not set, you should only use --saved flag. Exiting."
+        )
+        print("To fix: Add --train or --saved flag.")
+        sys.exit(1)

views_pipeline_core-0.1.0/views_pipeline_core/configs/drift_detection.py

@@ -0,0 +1,35 @@
+drift_detection_partition_dict={
+    'calibration':{
+            'global_missingness': {'threshold': 0.01},
+            'time_missingness':   {'threshold': 0.05},
+            'space_missingness':   {'threshold': 0.05},
+            'feature_missingness': {'threshold': 0.05},
+            'global_zeros': {'threshold': 0.99},
+            'time_zeros':   {'threshold': 0.95},
+            'feature_zeros': {'threshold': 0.95},
+            'standard_partition_length': 30,
+            'test_partition_length': 1
+
+             },
+    'testing':{
+        'global_missingness': {'threshold': 0.01},
+        'time_missingness': {'threshold': 0.05},
+        'space_missingness': {'threshold': 0.05},
+        'feature_missingness': {'threshold': 0.05},
+        'global_zeros': {'threshold': 0.99},
+        'time_zeros': {'threshold': 0.95},
+        'feature_zeros': {'threshold': 0.95},
+        'standard_partition_length': 30,
+        'test_partition_length': 1
+
+    },
+
+    'forecasting':{
+        'delta_completeness':   {'threshold': 0.01},
+        'delta_zeroes': {'threshold': 0.01},
+#        'ks_drift': {'threshold': 100},
+        'extreme_values': {'threshold': 5.0},
+        'standard_partition_length': 30,
+        'test_partition_length': 1
+    }
+}