validmind 2.5.6__py3-none-any.whl → 2.5.15__py3-none-any.whl

validmind/tests/model_validation/statsmodels/RegressionModelSummary.py

@@ -17,36 +17,38 @@ class RegressionModelSummary(Metric):
     """
     Evaluates regression model performance using metrics including R-Squared, Adjusted R-Squared, MSE, and RMSE.
 
-    **Purpose**: This metric test evaluates the performance of regression models by measuring their predictive ability
-    with regards to dependent variables given changes in the independent variables. Its measurement tools include
-    conventional regression metrics such as R-Squared, Adjusted R-Squared, Mean Squared Error (MSE), and Root Mean
-    Squared Error (RMSE).
-
-    **Test Mechanism**: This test employs the 'train_ds' attribute of the model to gather and analyze the training
-    data. Initially, it fetches the independent variables and uses the model to make predictions on these given
-    features. Subsequently, it calculates several standard regression performance metrics including R-Square, Adjusted
-    R-Squared, Mean Squared Error (MSE), and Root Mean Squared Error (RMSE), which quantify the approximation of the
-    predicted responses to the actual responses.
-
-    **Signs of High Risk**:
-    - Low R-Squared and Adjusted R-Squared values. A poor fit between the model predictions and the true responses is
-    indicated by low values of these metrics, suggesting the model explains a small fraction of the variance in the
-    target variable.
-    - High MSE and RMSE values represent a high prediction error and point to poor model performance.
-
-    **Strengths**:
+    ### Purpose
+
+    The Regression Model Summary test evaluates the performance of regression models by measuring their predictive
+    ability regarding dependent variables given changes in the independent variables. It uses conventional regression
+    metrics such as R-Squared, Adjusted R-Squared, Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) to
+    assess the model's accuracy and fit.
+
+    ### Test Mechanism
+
+    This test employs the 'train_ds' attribute of the model to gather and analyze the training data. Initially, it
+    fetches the independent variables and uses the model to make predictions on these given features. Subsequently, it
+    calculates several standard regression performance metrics including R-Squared, Adjusted R-Squared, Mean Squared
+    Error (MSE), and Root Mean Squared Error (RMSE), which quantify the approximation of the predicted responses to the
+    actual responses.
+
+    ### Signs of High Risk
+
+    - Low R-Squared and Adjusted R-Squared values.
+    - High MSE and RMSE values.
+
+    ### Strengths
+
     - Offers an extensive evaluation of regression models by combining four key measures of model accuracy and fit.
     - Provides a comprehensive view of the model's performance.
-    - Both the R-Squared and Adjusted R-Squared measures are readily interpretable. They represent the proportion of
-    total variation in the dependent variable that can be explained by the independent variables.
-
-    **Limitations**:
-    - Applicable exclusively to regression models. It is not suited for evaluating binary classification models or time
-    series models, thus limiting its scope.
-    - Although RMSE and MSE are sound measures of prediction error, they might be sensitive to outliers, potentially
-    leading to an overestimation of the model's prediction error.
-    - A high R-squared or adjusted R-squared may not necessarily indicate a good model, especially in cases where the
-    model is possibly overfitting the data.
+    - Both the R-Squared and Adjusted R-Squared measures are readily interpretable.
+
+    ### Limitations
+
+    - Applicable exclusively to regression models.
+    - RMSE and MSE might be sensitive to outliers.
+    - A high R-Squared or Adjusted R-Squared may not necessarily indicate a good model, especially in cases of
+    overfitting.
     """
 
     name = "regression_model_summary"

validmind/tests/model_validation/statsmodels/RegressionPermutationFeatureImportance.py

@@ -21,28 +21,35 @@ logger = get_logger(__name__)
 class RegressionPermutationFeatureImportance(Metric):
     """
     Assesses the significance of each feature in a model by evaluating the impact on model performance when feature
-    values are randomly rearranged. Specifically designed for use with statsmodels, this metric offers insight into the
-    importance of features based on the decrease in model's predictive accuracy, typically R².
+    values are randomly rearranged.
 
-    **Purpose**: The primary purpose of this metric is to determine which features significantly impact the performance
-    of a regression model developed using statsmodels. The metric measures how much the prediction accuracy deteriorates
+    ### Purpose
+
+    The primary purpose of this metric is to determine which features significantly impact the performance of a
+    regression model developed using statsmodels. The metric measures how much the prediction accuracy deteriorates
     when each feature's values are permuted.
 
-    **Test Mechanism**: This metric shuffles the values of each feature one at a time in the dataset, computes the model's
-    performance after each permutation, and compares it to the baseline performance. A significant decrease in performance
+    ### Test Mechanism
+
+    This metric shuffles the values of each feature one at a time in the dataset, computes the model's performance
+    after each permutation, and compares it to the baseline performance. A significant decrease in performance
     indicates the importance of the feature.
 
-    **Signs of High Risk**:
-    - Significant reliance on a feature that when permuted leads to a substantial decrease in performance, suggesting
+    ### Signs of High Risk
+
+    - Significant reliance on a feature that, when permuted, leads to a substantial decrease in performance, suggesting
     overfitting or high model dependency on that feature.
     - Features identified as unimportant despite known impacts from domain knowledge, suggesting potential issues in
     model training or data preprocessing.
 
-    **Strengths**:
-    - Directly assesses the impact of each feature on model performance, providing clear insights into model dependencies.
+    ### Strengths
+
+    - Directly assesses the impact of each feature on model performance, providing clear insights into model
+    dependencies.
     - Model-agnostic within the scope of statsmodels, applicable to any regression model that outputs predictions.
 
-    **Limitations**:
+    ### Limitations
+
     - The metric is specific to statsmodels and cannot be used with other types of models without adaptation.
     - It does not capture interactions between features, which can lead to underestimating the importance of correlated
     features.

validmind/tests/model_validation/statsmodels/RunsTest.py

@@ -11,41 +11,45 @@ class RunsTest(Metric):
     """
     Executes Runs Test on ML model to detect non-random patterns in output data sequence.
 
-    **Purpose**: The Runs Test is a statistical procedure used to determine whether the sequence of data extracted from
-    the ML model behaves randomly or not. Specifically, it analyzes runs, sequences of consecutive positives or
-    negatives, in the data to check if there are more or fewer runs than expected under the assumption of randomness.
-    This can be an indication of some pattern, trend, or cycle in the model's output which may need attention.
-
-    **Test Mechanism**: The testing mechanism applies the Runs Test from the statsmodels module on each column of the
-    training dataset. For every feature in the dataset, a Runs Test is executed, whose output includes a Runs Statistic
-    and P-value. A low P-value suggests that data arrangement in the feature is not likely to be random. The results
-    are stored in a dictionary where the keys are the feature names, and the values are another dictionary storing the
-    test statistic and the P-value for each feature.
-
-    **Signs of High Risk**:
+    ### Purpose
+
+    The Runs Test is a statistical procedure used to determine whether the sequence of data extracted from the ML model
+    behaves randomly or not. Specifically, it analyzes runs, sequences of consecutive positives or negatives, in the
+    data to check if there are more or fewer runs than expected under the assumption of randomness. This can be an
+    indication of some pattern, trend, or cycle in the model's output which may need attention.
+
+    ### Test Mechanism
+
+    The testing mechanism applies the Runs Test from the statsmodels module on each column of the training dataset. For
+    every feature in the dataset, a Runs Test is executed, whose output includes a Runs Statistic and P-value. A low
+    P-value suggests that data arrangement in the feature is not likely to be random. The results are stored in a
+    dictionary where the keys are the feature names, and the values are another dictionary storing the test statistic
+    and the P-value for each feature.
+
+    ### Signs of High Risk
+
     - High risk is indicated when the P-value is close to zero.
-    - If the p-value is less than a predefined significance level (like 0.05), it suggests that the runs (series of
+    - If the P-value is less than a predefined significance level (like 0.05), it suggests that the runs (series of
     positive or negative values) in the model's output are not random and are longer or shorter than what is expected
     under a random scenario.
     - This would mean there's a high risk of non-random distribution of errors or model outcomes, suggesting potential
     issues with the model.
 
-    **Strengths**:
-    - The strength of the Runs Test is that it's straightforward and fast for detecting non-random patterns in data
-    sequence.
-    - It can validate assumptions of randomness, which is particularly valuable for checking error distributions in
-    regression models, trendless time series data, and making sure a classifier doesn't favour one class over another.
-    - Moreover, it can be applied to both classification and regression tasks, making it versatile.
-
-    **Limitations**:
-    - The test assumes that the data is independently and identically distributed (i.i.d.), which might not be the case
-    for many real-world datasets.
-    - The conclusion drawn from the low p-value indicating non-randomness does not provide information about the type
+    ### Strengths
+
+    - Straightforward and fast for detecting non-random patterns in data sequence.
+    - Validates assumptions of randomness, which is valuable for checking error distributions in regression models,
+    trendless time series data, and ensuring a classifier doesn't favor one class over another.
+    - Can be applied to both classification and regression tasks, making it versatile.
+
+    ### Limitations
+
+    - Assumes that the data is independently and identically distributed (i.i.d.), which might not be the case for many
+    real-world datasets.
+    - The conclusion drawn from the low P-value indicating non-randomness does not provide information about the type
     or the source of the detected pattern.
-    - Also, it is sensitive to extreme values (outliers), and overly large or small run sequences can influence the
-    results.
-    - Furthermore, this test does not provide model performance evaluation; it is used to detect patterns in the
-    sequence of outputs only.
+    - Sensitive to extreme values (outliers), and overly large or small run sequences can influence the results.
+    - Does not provide model performance evaluation; it is used to detect patterns in the sequence of outputs only.
     """
 
     name = "runs_test"

validmind/tests/model_validation/statsmodels/ScorecardHistogram.py

@@ -2,136 +2,104 @@
 # See the LICENSE file in the root of this repository for details.
 # SPDX-License-Identifier: AGPL-3.0 AND ValidMind Commercial
 
-from dataclasses import dataclass
-
 import plotly.graph_objects as go
 from matplotlib import cm
 
-from validmind.vm_models import Figure, Metric
+from validmind import tags, tasks
 
 
-@dataclass
-class ScorecardHistogram(Metric):
+@tags("visualization", "credit_risk", "logistic_regression")
+@tasks("classification")
+def ScorecardHistogram(dataset, title="Histogram of Scores", score_column="score"):
     """
-    Creates histograms of credit scores, from both default and non-default instances, generated by a credit-risk model.
-
-    **Purpose**: The Scorecard Histogram test metric provides a visual interpretation of the credit scores generated by
-    a machine learning model for credit-risk classification tasks. It aims to compare the alignment of the model's
-    scoring decisions with the actual outcomes of credit loan applications. It helps in identifying potential
-    discrepancies between the model's predictions and real-world risk levels.
-
-    **Test Mechanism**: This metric uses logistic regression to generate a histogram of credit scores for both default
-    (negative class) and non-default (positive class) instances. Using both training and test datasets, the metric
-    calculates the credit score of each instance with a scorecard method, considering the impact of different features
-    on the likelihood of default. İncludes the default point to odds (PDO) scaling factor and predefined target score
-    and odds settings. Histograms for training and test sets are computed and plotted separately to offer insights into
-    the model's generalizability to unseen data.
-
-    **Signs of High Risk**:
-    - Discrepancies between the distributions of training and testing data, indicating a model's poor generalisation
+    The Scorecard Histogram test evaluates the distribution of credit scores between default and non-default instances,
+    providing critical insights into the performance and generalizability of credit-risk models.
+
+    ### Purpose
+
+    The Scorecard Histogram test metric provides a visual interpretation of the credit scores generated by a machine
+    learning model for credit-risk classification tasks. It aims to compare the alignment of the model's scoring
+    decisions with the actual outcomes of credit loan applications. It helps in identifying potential discrepancies
+    between the model's predictions and real-world risk levels.
+
+    ### Test Mechanism
+
+    This metric uses logistic regression to generate a histogram of credit scores for both default (negative class) and
+    non-default (positive class) instances. Using both training and test datasets, the metric calculates the credit
+    score of each instance with a scorecard method, considering the impact of different features on the likelihood of
+    default. It includes the default point to odds (PDO) scaling factor and predefined target score and odds settings.
+    Histograms for training and test sets are computed and plotted separately to offer insights into the model's
+    generalizability to unseen data.
+
+    ### Signs of High Risk
+
+    - Discrepancies between the distributions of training and testing data, indicating a model's poor generalization
     ability
-    - Skewed distributions favouring specific scores or classes, representing potential bias
+    - Skewed distributions favoring specific scores or classes, representing potential bias
+
+    ### Strengths
 
-    **Strengths**:
     - Provides a visual interpretation of the model's credit scoring system, enhancing comprehension of model behavior
     - Enables a direct comparison between actual and predicted scores for both training and testing data
     - Its intuitive visualization helps understand the model's ability to differentiate between positive and negative
     classes
     - Can unveil patterns or anomalies not easily discerned through numerical metrics alone
 
-    **Limitations**:
-    - Despite its value for visual interpretation, it doesn't quantify the performance of the model, and therefore may
+    ### Limitations
+
+    - Despite its value for visual interpretation, it doesn't quantify the performance of the model and therefore may
     lack precision for thorough model evaluation
     - The quality of input data can strongly influence the metric, as bias or noise in the data will affect both the
     score calculation and resultant histogram
     - Its specificity to credit scoring models limits its applicability across a wider variety of machine learning
     tasks and models
-    - The metric's effectiveness is somewhat tied to the subjective interpretation of the analyst, since it relies on
-    the analyst's judgment of the characteristics and implications of the plot.
+    - The metric's effectiveness is somewhat tied to the subjective interpretation of the analyst, relying on their
+    judgment of the characteristics and implications of the plot.
     """
 
-    name = "scorecard_histogram"
-    required_inputs = ["datasets"]
-    tasks = ["classification"]
-    tags = ["tabular_data", "visualization", "credit_risk"]
+    if score_column not in dataset.df.columns:
+        raise ValueError(
+            f"The required column '{score_column}' is not present in the dataset with input_id {dataset.input_id}"
+        )
 
-    default_params = {
-        "title": "Histogram of Scores",
-        "score_column": "score",
-    }
+    df = dataset.df
 
-    @staticmethod
-    def plot_score_histogram(dataframes, dataset_titles, score_col, target_col, title):
-        figures = []
-        # Generate a colormap and convert to Plotly-accepted color format
-        # Adjust 'viridis' to any other matplotlib colormap if desired
-        colormap = cm.get_cmap("viridis")
-
-        for _, (df, dataset_title) in enumerate(zip(dataframes, dataset_titles)):
-            fig = go.Figure()
-
-            # Get unique classes and assign colors
-            classes = sorted(df[target_col].unique())
-            colors = [
-                colormap(i / len(classes))[:3] for i in range(len(classes))
-            ]  # RGB
-            color_dict = {
-                cls: f"rgb({int(rgb[0]*255)}, {int(rgb[1]*255)}, {int(rgb[2]*255)})"
-                for cls, rgb in zip(classes, colors)
-            }
-
-            for class_value in sorted(df[target_col].unique()):
-                scores_class = df[df[target_col] == class_value][score_col]
-                fig.add_trace(
-                    go.Histogram(
-                        x=scores_class,
-                        opacity=0.75,
-                        name=f"{dataset_title} {target_col} = {class_value}",
-                        marker=dict(
-                            color=color_dict[class_value],
-                        ),
-                    )
-                )
-            fig.update_layout(
-                barmode="overlay",
-                title_text=f"{title} - {dataset_title}",
-                xaxis_title="Score",
-                yaxis_title="Frequency",
-                legend_title=target_col,
-            )
-            figures.append(fig)
-        return figures
-
-    def run(self):
-        title = self.params["title"]
-        score_column = self.params["score_column"]
-        dataset_titles = [dataset.input_id for dataset in self.inputs.datasets]
-        target_column = self.inputs.datasets[0].target_column
-
-        dataframes = []
-        metric_value = {"score_histogram": {}}
-        for dataset in self.inputs.datasets:
-            if score_column not in dataset.df.columns:
-                raise ValueError(
-                    f"The required column '{score_column}' is not present in the dataset with input_id {dataset.input_id}"
-                )
-
-            dataframes.append(dataset.df.copy())
-            metric_value["score_histogram"][dataset.input_id] = list(
-                dataset.df[score_column]
-            )
+    fig = _plot_score_histogram(df, score_column, dataset.target_column, title)
 
-        figures = self.plot_score_histogram(
-            dataframes, dataset_titles, score_column, target_column, title
-        )
+    return fig
 
-        figures_list = [
-            Figure(
-                for_object=self,
-                key=f"score_histogram_{title.replace(' ', '_')}_{i+1}",
-                figure=fig,
-            )
-            for i, fig in enumerate(figures)
-        ]
 
-        return self.cache_results(metric_value=metric_value, figures=figures_list)
+def _plot_score_histogram(df, score_col, target_col, title):
+    # Generate a colormap and convert to Plotly-accepted color format
+    # Adjust 'viridis' to any other matplotlib colormap if desired
+    colormap = cm.get_cmap("viridis")
+
+    fig = go.Figure()
+
+    # Get unique classes and assign colors
+    classes = sorted(df[target_col].unique())
+    colors = [colormap(i / len(classes))[:3] for i in range(len(classes))]  # RGB
+    color_dict = {
+        cls: f"rgb({int(rgb[0]*255)}, {int(rgb[1]*255)}, {int(rgb[2]*255)})"
+        for cls, rgb in zip(classes, colors)
+    }
+
+    for class_value in sorted(df[target_col].unique()):
+        scores_class = df[df[target_col] == class_value][score_col]
+        fig.add_trace(
+            go.Histogram(
+                x=scores_class,
+                opacity=0.75,
+                name=f"{target_col} = {class_value}",
+                marker=dict(
+                    color=color_dict[class_value],
+                ),
+            )
+        )
+        fig.update_layout(
+            barmode="overlay",
+            title_text=f"{title}",
+            xaxis_title="Score",
+            yaxis_title="Frequency",
+        )
+    return fig

validmind/tests/model_validation/statsmodels/ShapiroWilk.py

@@ -11,29 +11,36 @@ class ShapiroWilk(Metric):
     """
     Evaluates feature-wise normality of training data using the Shapiro-Wilk test.
 
-    **Purpose**: The Shapiro-Wilk test is utilized to investigate whether a particular dataset conforms to the standard
-    normal distribution. This analysis is crucial in machine learning modeling because the normality of the data can
+    ### Purpose
+
+    The Shapiro-Wilk test is utilized to investigate whether a particular dataset conforms to the standard normal
+    distribution. This analysis is crucial in machine learning modeling because the normality of the data can
     profoundly impact the performance of the model. This metric is especially useful in evaluating various features of
     the dataset in both classification and regression tasks.
 
-    **Test Mechanism**: The Shapiro-Wilk test is conducted on each feature column of the training dataset to determine
-    if the data contained fall within the normal distribution. The test presents a statistic and a p-value, with the
-    p-value serving to validate or repudiate the null hypothesis, which is that the tested data is normally distributed.
+    ### Test Mechanism
+
+    The Shapiro-Wilk test is conducted on each feature column of the training dataset to determine if the data
+    contained fall within the normal distribution. The test presents a statistic and a p-value, with the p-value
+    serving to validate or repudiate the null hypothesis, which is that the tested data is normally distributed.
+
+    ### Signs of High Risk
 
-    **Signs of High Risk**:
     - A p-value that falls below 0.05 signifies a high risk as it discards the null hypothesis, indicating that the
     data does not adhere to the normal distribution.
     - For machine learning models built on the presumption of data normality, such an outcome could result in subpar
     performance or incorrect predictions.
 
-    **Strengths**:
+    ### Strengths
+
     - The Shapiro-Wilk test is esteemed for its level of accuracy, thereby making it particularly well-suited to
     datasets of small to moderate sizes.
     - It proves its versatility through its efficient functioning in both classification and regression tasks.
     - By separately testing each feature column, the Shapiro-Wilk test can raise an alarm if a specific feature does
     not comply with the normality.
 
-    **Limitations**:
+    ### Limitations
+
     - The Shapiro-Wilk test's sensitivity can be a disadvantage as it often rejects the null hypothesis (i.e., data is
     normally distributed), even for minor deviations, especially in large datasets. This may lead to unwarranted 'false
     alarms' of high risk by deeming the data as not normally distributed even if it approximates normal distribution.

validmind/tests/ongoing_monitoring/FeatureDrift.py

@@ -16,37 +16,41 @@ def FeatureDrift(
     datasets, bins=[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9], feature_columns=None
 ):
     """
-    **Purpose**:
+    Evaluates changes in feature distribution over time to identify potential model drift.
+
+    ### Purpose
 
     The Feature Drift test aims to evaluate how much the distribution of features has shifted over time between two
     datasets, typically training and monitoring datasets. It uses the Population Stability Index (PSI) to quantify this
-    change, providing insights into the model's robustness and the necessity for retraining or feature engineering.
+    change, providing insights into the model’s robustness and the necessity for retraining or feature engineering.
 
-    **Test Mechanism**:
+    ### Test Mechanism
 
     This test calculates the PSI by:
+
     - Bucketing the distributions of each feature in both datasets.
     - Comparing the percentage of observations in each bucket between the two datasets.
     - Aggregating the differences across all buckets for each feature to produce the PSI score for that feature.
 
     The PSI score is interpreted as:
+
     - PSI < 0.1: No significant population change.
     - PSI < 0.2: Moderate population change.
     - PSI >= 0.2: Significant population change.
 
-    **Signs of High Risk**:
+    ### Signs of High Risk
 
     - PSI >= 0.2 for any feature, indicating a significant distribution shift.
     - Consistently high PSI scores across multiple features.
     - Sudden spikes in PSI in recent monitoring data compared to historical data.
 
-    **Strengths**:
+    ### Strengths
 
     - Provides a quantitative measure of feature distribution changes.
     - Easily interpretable thresholds for decision-making.
     - Helps in early detection of data drift, prompting timely interventions.
 
-    **Limitations**:
+    ### Limitations
 
     - May not capture more intricate changes in data distribution nuances.
     - Assumes that bucket thresholds (quantiles) adequately represent distribution shifts.

validmind/tests/ongoing_monitoring/PredictionAcrossEachFeature.py

@@ -12,31 +12,37 @@ from validmind import tags, tasks
 @tasks("monitoring")
 def PredictionAcrossEachFeature(datasets, model):
     """
-    **Purpose:**
-    This test shows visually the prediction using reference data and monitoring data across each individual feature. If
-    there are significant differences in predictions across feature values from reference to monitoring dataset, then
-    further investigation is needed as the model is producing predictions that are different than what was observed
-    during the training of the model.
-
-    **Test Mechanism:**
-    The test creates scatter plots for each feature, comparing the reference dataset (used for training) with the
-    monitoring dataset (used in production). Each plot has two subplots: one for the reference data and one for the
-    monitoring data, visualizing the prediction probabilities. This allows for a visual comparison of the model's
-    behavior across different datasets.
-
-    **Signs of High Risk:**
-    - Significant discrepancies between the reference and monitoring subplots for the same feature
-    - Unexpected patterns or trends in monitoring data that weren't present in reference data
-
-    **Strengths:**
-    - Provides a clear visual representation of model performance across different features
-    - Allows for easy identification of features where the model's predictions have changed
-    - Facilitates quick detection of potential issues with the model when deployed in production
-
-    **Limitations:**
-    - Interpretation of scatter plots can be subjective and may require expertise
-    - Visualizations do not provide quantitative metrics for objective evaluation
-    - May not capture all types of distribution changes or issues with the model's predictions
+    Assesses differences in model predictions across individual features between reference and monitoring datasets
+    through visual analysis.
+
+    ### Purpose
+
+    The Prediction Across Each Feature test aims to visually compare model predictions for each feature between
+    reference (training) and monitoring (production) datasets. It helps identify significant differences in prediction
+    patterns for further investigation and ensures the model's consistency and stability over time.
+
+    ### Test Mechanism
+
+    The test generates scatter plots for each feature, comparing prediction probabilities between the reference and
+    monitoring datasets. Each plot consists of two subplots: one for reference data and one for monitoring data,
+    enabling visual comparison of the model's predictive behavior.
+
+    ### Signs of High Risk
+
+    - Significant discrepancies between the reference and monitoring subplots for the same feature.
+    - Unexpected patterns or trends in monitoring data that were absent in reference data.
+
+    ### Strengths
+
+    - Provides a clear visual representation of model performance across different features.
+    - Facilitates easy identification of features where the model's predictions have diverged.
+    - Enables quick detection of potential model performance issues in production.
+
+    ### Limitations
+
+    - Interpretation of scatter plots can be subjective and may require expertise.
+    - Visualizations do not provide quantitative metrics for objective evaluation.
+    - May not capture all types of distribution changes or issues with the model's predictions.
     """
 
     """

validmind/tests/ongoing_monitoring/PredictionCorrelation.py

@@ -13,30 +13,38 @@ from validmind import tags, tasks
 @tasks("monitoring")
 def PredictionCorrelation(datasets, model):
     """
-    **Purpose:**
-    The test is used to assess the correlation pairs for each feature between model predictions from reference and
-    monitoring datasets. The primary goal is to detect significant changes in these pairs, which may signal target
-    drift, leading to lower model performance.
+    Assesses correlation changes between model predictions from reference and monitoring datasets to detect potential
+    target drift.
 
-    **Test Mechanism:**
-    The test calculates the correlation of each feature with model predictions for both reference and monitoring
-    datasets. The test then compares these correlations side-by-side via a bar plot and a correlation table. Features
-    with significant changes in correlation pairs highlight potential risks of model drift.
+    ### Purpose
+
+    To evaluate the changes in correlation pairs between model predictions and features from reference and monitoring
+    datasets. This helps in identifying significant shifts that may indicate target drift, potentially affecting model
+    performance.
+
+    ### Test Mechanism
+
+    This test calculates the correlation of each feature with model predictions for both reference and monitoring
+    datasets. It then compares these correlations side-by-side using a bar plot and a correlation table. Significant
+    changes in correlation pairs are highlighted to signal possible model drift.
+
+    ### Signs of High Risk
 
-    **Signs of High Risk:**
     - Significant changes in correlation pairs between the reference and monitoring predictions.
-    - Notable correlation differences indicating a potential shift in the relationship between features and the target
-    variable.
-
-    **Strengths:**
-    - Allows for visual identification of drift in feature relationships with model predictions.
-    - Comparison via a clear bar plot assists in understanding model stability over time.
-    - Helps in early detection of target drift, enabling timely interventions.
-
-    **Limitations:**
-    - May require substantial reference and monitoring data for accurate comparison.
-    - Correlation does not imply causation, and other factors might influence changes.
-    - The method solely focuses on linear relationships, potentially missing non-linear interactions.
+    - Notable differences in correlation values, indicating a possible shift in the relationship between features and
+    the target variable.
+
+    ### Strengths
+
+    - Provides visual identification of drift in feature relationships with model predictions.
+    - Clear bar plot comparison aids in understanding model stability over time.
+    - Enables early detection of target drift, facilitating timely interventions.
+
+    ### Limitations
+
+    - Requires substantial reference and monitoring data for accurate comparison.
+    - Correlation does not imply causation; other factors may influence changes.
+    - Focuses solely on linear relationships, potentially missing non-linear interactions.
     """
 
     prediction_prob_column = f"{model.input_id}_probabilities"