validmind 2.5.8__py3-none-any.whl → 2.5.15__py3-none-any.whl

validmind/tests/data_validation/nlp/Hashtags.py

@@ -20,37 +20,44 @@ class Hashtags(ThresholdTest):
     """
     Assesses hashtag frequency in a text column, highlighting usage trends and potential dataset bias or spam.
 
-    **Purpose**: The Hashtags test is designed to measure the frequency of hashtags used within a given text column in
-    a dataset. It is particularly useful for natural language processing tasks such as text classification and text
-    summarization. The goal is to identify common trends and patterns in the use of hashtags, which can serve as
-    critical indicators or features within a machine learning model.
+    ### Purpose
 
-    **Test Mechanism**: The test implements a regular expression (regex) to extract all hashtags from the specified
-    text column. For each hashtag found, it makes a tally of its occurrences. It then outputs a list of the top N
-    hashtags (default is 25, but customizable), sorted by their counts in descending order. The results are also
-    visualized in a bar plot, with frequency counts on the y-axis and the corresponding hashtags on the x-axis.
+    The Hashtags test is designed to measure the frequency of hashtags used within a given text column in a dataset. It
+    is particularly useful for natural language processing tasks such as text classification and text summarization.
+    The goal is to identify common trends and patterns in the use of hashtags, which can serve as critical indicators
+    or features within a machine learning model.
+
+    ### Test Mechanism
+
+    The test implements a regular expression (regex) to extract all hashtags from the specified text column. For each
+    hashtag found, it makes a tally of its occurrences. It then outputs a list of the top N hashtags (default is 25,
+    but customizable), sorted by their counts in descending order. The results are also visualized in a bar plot, with
+    frequency counts on the y-axis and the corresponding hashtags on the x-axis.
+
+    ### Signs of High Risk
 
-    **Signs of High Risk**:
     - A low diversity in the usage of hashtags, as indicated by a few hashtags being used disproportionately more than
     others.
     - Repeated usage of one or few hashtags can be indicative of spam or a biased dataset.
     - If there are no or extremely few hashtags found in the dataset, it perhaps signifies that the text data does not
     contain structured social media data.
 
-    **Strengths**:
-    - It provides a concise visual representation of the frequency of hashtags, which can be critical for understanding
+    ### Strengths
+
+    - Provides a concise visual representation of the frequency of hashtags, which can be critical for understanding
     trends about a particular topic in text data.
-    - It is instrumental in tasks specifically related to social media text analytics, such as opinion analysis and
-    trend discovery.
-    - The test is adaptable, allowing the flexibility to determine the number of top hashtags to be analyzed.
+    - Instrumental in tasks specifically related to social media text analytics, such as opinion analysis and trend
+    discovery.
+    - Adaptable, allowing the flexibility to determine the number of top hashtags to be analyzed.
+
+    ### Limitations
 
-    **Limitations**:
-    - The test assumes the presence of hashtags and therefore may not be applicable for text datasets that do not
-    contain hashtags (e.g., formal documents, scientific literature).
+    - Assumes the presence of hashtags and therefore may not be applicable for text datasets that do not contain
+    hashtags (e.g., formal documents, scientific literature).
     - Language-specific limitations of hashtag formulations are not taken into account.
-    - It does not account for typographical errors, variations, or synonyms in hashtags.
-    - This test does not provide context or sentiment associated with the hashtags, so the information provided may
-    have limited utility on its own.
+    - Does not account for typographical errors, variations, or synonyms in hashtags.
+    - Does not provide context or sentiment associated with the hashtags, so the information provided may have limited
+    utility on its own.
     """
 
     name = "hashtags"

validmind/tests/data_validation/nlp/LanguageDetection.py

@@ -17,24 +17,43 @@ from validmind import tags, tasks
 @tasks("text_classification", "text_summarization")
 def LanguageDetection(dataset):
     """
-    Detects the language of each text entry in a dataset and visualizes the distribution of languages
-    as a histogram.
+    Assesses the diversity of languages in a textual dataset by detecting and visualizing the distribution of languages.
 
-    This method checks for a specified text column in the dataset's dataframe, uses a language detection
-    library to determine the language of each text entry, and returns a histogram plot of the language
-    distribution.
+    ### Purpose
 
-    Args:
-        dataset (Dataset): A dataset object which must have a `df` attribute (a pandas DataFrame)
-            and a `text_column` attribute indicating the name of the column containing text. If the
-            `text_column` attribute is not set, a ValueError is raised.
+    The Language Detection test aims to identify and visualize the distribution of languages present within a textual
+    dataset. This test helps in understanding the diversity of languages in the data, which is crucial for developing
+    and validating multilingual models.
 
-    Returns:
-        plotly.graph_objs._figure.Figure: A Plotly histogram plot showing the distribution of detected
-        languages across the dataset's text entries.
+    ### Test Mechanism
 
-    Raises:
-        ValueError: If the `text_column` is not specified in the dataset object.
+    This test operates by:
+
+    - Checking if the dataset has a specified text column.
+    - Using a language detection library to determine the language of each text entry in the dataset.
+    - Generating a histogram plot of the language distribution, with language codes on the x-axis and their frequencies
+    on the y-axis.
+
+    If the text column is not specified, a ValueError is raised to ensure proper dataset configuration.
+
+    ### Signs of High Risk
+
+    - A high proportion of entries returning "Unknown" language codes.
+    - Detection of unexpectedly diverse or incorrect language codes, indicating potential data quality issues.
+    - Significant imbalance in language distribution, which might indicate potential biases in the dataset.
+
+    ### Strengths
+
+    - Provides a visual representation of language diversity within the dataset.
+    - Helps identify data quality issues related to incorrect or unknown language detection.
+    - Useful for ensuring that multilingual models have adequate and appropriate representation from various languages.
+
+    ### Limitations
+
+    - Dependency on the accuracy of the language detection library, which may not be perfect.
+    - Languages with similar structures or limited text length may be incorrectly classified.
+    - The test returns "Unknown" for entries where language detection fails, which might mask underlying issues with
+    certain languages or text formats.
     """
     # check text column
     if not dataset.text_column:

validmind/tests/data_validation/nlp/Mentions.py

@@ -20,31 +20,37 @@ class Mentions(ThresholdTest):
     """
     Calculates and visualizes frequencies of '@' prefixed mentions in a text-based dataset for NLP model analysis.
 
-    **Purpose**: This test, termed "Mentions", is designed to gauge the quality of data in a Natural Language
-    Processing (NLP) or text-focused Machine Learning model. The primary objective is to identify and calculate the
-    frequency of 'mentions' within a chosen text column of a dataset. A 'mention' in this context refers to individual
-    text elements that are prefixed by '@'. The output of this test reveals the most frequently mentioned entities or
-    usernames, which can be integral for applications such as social media analyses, customer sentiment analyses, and
-    so on.
-
-    **Test Mechanism**: The test first verifies the existence of a text column in the provided dataset. It then employs
-    a regular expression pattern to extract mentions from the text. Subsequently, the frequency of each unique mention
-    is calculated. The test selects the most frequent mentions based on default or user-defined parameters, the default
+    ### Purpose
+
+    The "Mentions" test is designed to gauge the quality of data in a Natural Language Processing (NLP) or text-focused
+    Machine Learning model. The primary objective is to identify and calculate the frequency of 'mentions' within a
+    chosen text column of a dataset. A 'mention' in this context refers to individual text elements that are prefixed
+    by '@'. The output of this test reveals the most frequently mentioned entities or usernames, which can be integral
+    for applications such as social media analyses or customer sentiment analyses.
+
+    ### Test Mechanism
+
+    The test first verifies the existence of a text column in the provided dataset. It then employs a regular
+    expression pattern to extract mentions from the text. Subsequently, the frequency of each unique mention is
+    calculated. The test selects the most frequent mentions based on default or user-defined parameters, the default
     being the top 25, for representation. This process of thresholding forms the core of the test. A treemap plot
     visualizes the test results, where the size of each rectangle corresponds to the frequency of a particular mention.
 
-    **Signs of High Risk**:
+    ### Signs of High Risk
+
     - The lack of a valid text column in the dataset, which would result in the failure of the test execution.
     - The absence of any mentions within the text data, indicating that there might not be any text associated with
-    '@'. This situation could point towards sparse or poor-quality data, thereby hampering the model's generalization
-    or learning capabilities.
+    '@'. This situation could point toward sparse or poor-quality data, thereby hampering the model's generalization or
+    learning capabilities.
+
+    ### Strengths
 
-    **Strengths**:
     - The test is specifically optimized for text-based datasets which gives it distinct power in the context of NLP.
     - It enables quick identification and visually appealing representation of the predominant elements or mentions.
     - It can provide crucial insights about the most frequently mentioned entities or usernames.
 
-    **Limitations**:
+    ### Limitations
+
     - The test only recognizes mentions that are prefixed by '@', hence useful textual aspects not preceded by '@'
     might be ignored.
     - This test isn't suited for datasets devoid of textual data.

validmind/tests/data_validation/nlp/PolarityAndSubjectivity.py

@@ -14,18 +14,41 @@ from validmind import tags, tasks
 @tasks("nlp")
 def PolarityAndSubjectivity(dataset):
     """
-    Analyzes the polarity and subjectivity of text data within a dataset.
+    Analyzes the polarity and subjectivity of text data within a given dataset to visualize the sentiment distribution.
 
-    This method processes a dataset containing textual data to compute the polarity and
-    subjectivity scores using TextBlob, and returns a Plotly scatter plot visualizing
-    these scores.
+    ### Purpose
 
-    Args:
-        dataset (Dataset): A dataset object which must have a `df` attribute (a pandas DataFrame)
-            and a `text_column` attribute indicating the name of the column containing text.
+    The Polarity and Subjectivity test is designed to evaluate the sentiment expressed in textual data. By analyzing
+    these aspects, it helps to identify the emotional tone and subjectivity of the dataset, which could be crucial in
+    understanding customer feedback, social media sentiments, or other text-related data.
 
-    Returns:
-        plotly.graph_objs._figure.Figure: A Plotly scatter plot of polarity vs subjectivity.
+    ### Test Mechanism
+
+    This test uses TextBlob to compute the polarity and subjectivity scores of textual data in a given dataset. The
+    mechanism includes:
+
+    - Iterating through each text entry in the specified column of the dataset.
+    - Applying the TextBlob library to compute the polarity (ranging from -1 for negative sentiment to +1 for positive
+    sentiment) and subjectivity (ranging from 0 for objective to 1 for subjective) for each entry.
+    - Creating a scatter plot using Plotly to visualize the relationship between polarity and subjectivity.
+
+    ### Signs of High Risk
+
+    - High concentration of negative polarity values indicating prevalent negative sentiments.
+    - High subjectivity scores suggesting the text data is largely opinion-based rather than factual.
+    - Disproportionate clusters of extreme scores (e.g., many points near -1 or +1 polarity).
+
+    ### Strengths
+
+    - Quantifies sentiment and subjectivity which can provide actionable insights.
+    - Visualizes sentiment distribution, aiding in easy interpretation.
+    - Utilizes well-established TextBlob library for sentiment analysis.
+
+    ### Limitations
+
+    - Polarity and subjectivity calculations may oversimplify nuanced text sentiments.
+    - Reliance on TextBlob which may not be accurate for all domains or contexts.
+    - Visualization could become cluttered with very large datasets, making interpretation difficult.
     """
 
     # Function to calculate sentiment and subjectivity

validmind/tests/data_validation/nlp/Punctuations.py

@@ -20,34 +20,38 @@ class Punctuations(Metric):
     """
     Analyzes and visualizes the frequency distribution of punctuation usage in a given text dataset.
 
-    **1. Purpose:** The Punctuations Metric's primary purpose is to analyze the frequency of punctuation usage within a
-    given text dataset. This is often used in Natural Language Processing tasks, such as text classification and text
+    ### Purpose
+
+    The Punctuations Metric's primary purpose is to analyze the frequency of punctuation usage within a given text
+    dataset. This is often used in Natural Language Processing tasks, such as text classification and text
     summarization.
 
-    **2. Test Mechanism:** The test begins by verifying that the input "dataset" is of the type VMDataset. Following
-    that, a corpus is created from the dataset by splitting its text on spaces. Each unique punctuation character in
-    the text corpus is then tallied. Then, the frequency distribution of each punctuation symbol is visualized as a bar
-    graph, with these results being stored as Figures and associated with the main Punctuations object.
+    ### Test Mechanism
+
+    The test begins by verifying that the input "dataset" is of the type VMDataset. Following that, a corpus is created
+    from the dataset by splitting its text on spaces. Each unique punctuation character in the text corpus is then
+    tallied. The frequency distribution of each punctuation symbol is visualized as a bar graph, with these results
+    being stored as Figures and associated with the main Punctuations object.
 
-    **3. Signs of High Risk:**
+    ### Signs of High Risk
 
-    - High risk can be indicated by the excessive or unusual frequency of specific punctuation marks, potentially
-    denoting dubious quality, data corruption, or skewed data.
+    - Excessive or unusual frequency of specific punctuation marks, potentially denoting dubious quality, data
+    corruption, or skewed data.
 
-    **4. Strengths:**
+    ### Strengths
 
-    - The Punctuations Metric provides valuable insights into the distribution of punctuation usage in a text dataset.
-    - This insight can be important in validating the quality, consistency, and nature of the data.
-    - It can provide hints about the style or tonality of the text corpus. For example, frequent usage of exclamation
-    marks may suggest a more informal and emotional context.
+    - Provides valuable insights into the distribution of punctuation usage in a text dataset.
+    - Important in validating the quality, consistency, and nature of the data.
+    - Can provide hints about the style or tonality of the text corpus, such as informal and emotional context
+    indicated by frequent exclamation marks.
 
-    **5. Limitations:**
+    ### Limitations
 
-    - The metric focuses solely on punctuation usage and can miss other important textual characteristics.
-    - It's important not to make general cultural or tonality assumptions based solely on punctuation distribution,
-    since these can vary greatly across different languages and contexts.
-    - The metric may be less effective with languages that use non-standard or different punctuation.
-    - The visualization may lack interpretability when there are many unique punctuation marks in the dataset.
+    - Focuses solely on punctuation usage, potentially missing other important textual characteristics.
+    - General cultural or tonality assumptions based on punctuation distribution can be misguiding, as these vary
+    across different languages and contexts.
+    - Less effective with languages that use non-standard or different punctuation.
+    - Visualization may lack interpretability when there are many unique punctuation marks in the dataset.
     """
 
     name = "punctuations"

validmind/tests/data_validation/nlp/Sentiment.py

@@ -17,16 +17,35 @@ def Sentiment(dataset):
     """
     Analyzes the sentiment of text data within a dataset using the VADER sentiment analysis tool.
 
-    This method initializes the VADER SentimentIntensityAnalyzer and applies it to each text entry
-    in the specified column of the dataset's dataframe. It returns a KDE plot visualizing the distribution
-    of sentiment scores across the dataset.
+    ### Purpose
 
-    Args:
-        dataset (Dataset): A dataset object which must have a `df` attribute (a pandas DataFrame)
-            and a `text_column` attribute indicating the name of the column containing text.
+    The Sentiment test evaluates the overall sentiment of text data within a dataset. By analyzing sentiment scores, it
+    aims to ensure that the model is interpreting text data accurately and is not biased towards a particular sentiment.
 
-    Returns:
-        matplotlib.figure.Figure: A KDE plot visualizing the distribution of sentiment scores.
+    ### Test Mechanism
+
+    This test uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) SentimentIntensityAnalyzer. It processes
+    each text entry in a specified column of the dataset to calculate the compound sentiment score, which represents
+    the overall sentiment polarity. The distribution of these sentiment scores is then visualized using a KDE (Kernel
+    Density Estimation) plot, highlighting any skewness or concentration in sentiment.
+
+    ### Signs of High Risk
+
+    - Extreme polarity in sentiment scores, indicating potential bias.
+    - Unusual concentration of sentiment scores in a specific range.
+    - Significant deviation from expected sentiment distribution for the given text data.
+
+    ### Strengths
+
+    - Provides a clear visual representation of sentiment distribution.
+    - Uses a well-established sentiment analysis tool (VADER).
+    - Can handle a wide range of text data, making it flexible for various applications.
+
+    ### Limitations
+
+    - May not capture nuanced or context-specific sentiments.
+    - Relies heavily on the accuracy of the VADER sentiment analysis tool.
+    - Visualization alone may not provide comprehensive insights into underlying causes of sentiment distribution.
     """
     nltk.download("vader_lexicon", quiet=True)
     # Initialize VADER

validmind/tests/data_validation/nlp/StopWords.py

@@ -30,40 +30,47 @@ class StopWords(ThresholdTest):
     """
     Evaluates and visualizes the frequency of English stop words in a text dataset against a defined threshold.
 
-    **Purpose**: The StopWords threshold test is a tool designed for assessing the quality of text data in an ML model.
-    It focuses on the identification and analysis of "stop words" in a given dataset. Stop words are frequent, common,
-    yet semantically insignificant words (for example: "the", "and", "is") in a language. This test evaluates the
+    ### Purpose
+
+    The StopWords threshold test is a tool designed for assessing the quality of text data in an ML model. It focuses
+    on the identification and analysis of "stop words" in a given dataset. Stop words are frequent, common, yet
+    semantically insignificant words (for example: "the", "and", "is") in a language. This test evaluates the
     proportion of stop words to the total word count in the dataset, in essence, scrutinizing the frequency of stop
     word usage. The core objective is to highlight the prevalent stop words based on their usage frequency, which can
     be instrumental in cleaning the data from noise and improving ML model performance.
 
-    **Test Mechanism**: The StopWords test initiates on receiving an input of a 'VMDataset' object. Absence of such an
-    object will trigger an error. The methodology involves inspection of the text column of the VMDataset to create a
-    'corpus' (a collection of written texts). Leveraging the Natural Language Toolkit's (NLTK) stop word repository,
-    the test screens the corpus for any stop words and documents their frequency. It further calculates the percentage
-    usage of each stop word compared to the total word count in the corpus. This percentage is evaluated against a
-    predefined 'min_percent_threshold'. If this threshold is breached, the test returns a failed output. Top prevailing
-    stop words along with their usage percentages are returned, facilitated by a bar chart visualization of these stop
-    words and their frequency.
+    ### Test Mechanism
+
+    The StopWords test initiates on receiving an input of a 'VMDataset' object. Absence of such an object will trigger
+    an error. The methodology involves inspection of the text column of the VMDataset to create a 'corpus' (a
+    collection of written texts). Leveraging the Natural Language Toolkit's (NLTK) stop word repository, the test
+    screens the corpus for any stop words and documents their frequency. It further calculates the percentage usage of
+    each stop word compared to the total word count in the corpus. This percentage is evaluated against a predefined
+    'min_percent_threshold'. If this threshold is breached, the test returns a failed output. Top prevailing stop words
+    along with their usage percentages are returned, facilitated by a bar chart visualization of these stop words and
+    their frequency.
+
+    ### Signs of High Risk
 
-    **Signs of High Risk**:
     - A percentage of any stop words exceeding the predefined 'min_percent_threshold'.
     - High frequency of stop words in the dataset which may adversely affect the application's analytical performance
     due to noise creation.
 
-    **Strengths**:
+    ### Strengths
+
     - The ability to scrutinize and quantify the usage of stop words.
-    - Provides insights into potential noise in the text data due to stop words. This can directly aid in enhancing
-    model training efficiency.
-    - The test includes a bar chart visualization feature to easily interpret and action upon the stop words frequency
+    - Provides insights into potential noise in the text data due to stop words.
+    - Directly aids in enhancing model training efficiency.
+    - Includes a bar chart visualization feature to easily interpret and action upon the stop words frequency
     information.
 
-    **Limitations**:
+    ### Limitations
+
     - The test only supports English stop words, making it less effective with datasets of other languages.
     - The 'min_percent_threshold' parameter may require fine-tuning for different datasets, impacting the overall
     effectiveness of the test.
-    - Contextual use of the stop words within the dataset is not considered which may lead to overlooking their
-    significance in certain contexts.
+    - Contextual use of the stop words within the dataset is not considered, potentially overlooking their significance
+    in certain contexts.
     - The test focuses specifically on the frequency of stop words, not providing direct measures of model performance
     or predictive accuracy.
     """

validmind/tests/data_validation/nlp/TextDescription.py

@@ -17,46 +17,47 @@ from ....vm_models import Figure, Metric, VMDataset
 @dataclass
 class TextDescription(Metric):
     """
-    Performs comprehensive textual analysis on a dataset using NLTK, evaluating various parameters and generating
+    Conducts comprehensive textual analysis on a dataset using NLTK to evaluate various parameters and generate
     visualizations.
 
-    **Purpose**: This test uses the TextDescription metric to conduct a comprehensive textual analysis of a given
-    dataset. Various parameters such as total words, total sentences, average sentence length, total paragraphs, total
-    unique words, most common words, total punctuations, and lexical diversity are evaluated. This metric aids in
-    comprehending the nature of the text and evaluating the potential challenges that machine learning algorithms
-    deployed for textual analysis, language processing, or summarization might face.
-
-    **Test Mechanism**: The test works by parsing the given dataset and utilizes the NLTK (Natural Language Toolkit)
-    library for tokenizing the text into words, sentences, and paragraphs. Subsequently, it processes the text further
-    by eliminating stopwords declared in 'unwanted_tokens' and punctuations. Next, it determines parameters like the
-    total count of words, sentences, paragraphs, punctuations alongside the average sentence length and lexical
-    diversity. Lastly, the result from these calculations is condensed and scatter plots for certain variable
-    combinations (e.g. Total Words vs Total Sentences, Total Words vs Total Unique Words) are produced, providing a
-    visual representation of the text's structure.
-
-    **Signs of High Risk**:
-    - Anomalies or an increase in complexity within the lexical diversity results.
+    ### Purpose
+
+    The TextDescription test aims to conduct a thorough textual analysis of a dataset using the NLTK (Natural Language
+    Toolkit) library. It evaluates various metrics such as total words, total sentences, average sentence length, total
+    paragraphs, total unique words, most common words, total punctuations, and lexical diversity. The goal is to
+    understand the nature of the text and anticipate challenges machine learning models might face in text processing,
+    language understanding, or summarization tasks.
+
+    ### Test Mechanism
+
+    The test works by:
+
+    - Parsing the dataset and tokenizing the text into words, sentences, and paragraphs using NLTK.
+    - Removing stopwords and unwanted tokens.
+    - Calculating parameters like total words, total sentences, average sentence length, total paragraphs, total unique
+    words, total punctuations, and lexical diversity.
+    - Generating scatter plots to visualize correlations between various metrics (e.g., Total Words vs Total Sentences).
+
+    ### Signs of High Risk
+
+    - Anomalies or increased complexity in lexical diversity.
     - Longer sentences and paragraphs.
     - High uniqueness of words.
-    - Presence of a significant amount of unwanted tokens.
+    - Large number of unwanted tokens.
     - Missing or erroneous visualizations.
-     These signs suggest potential risk in text processing ML models, indicating that the ability of the model to
-    absorb and process text could be compromised.
-
-    **Strengths**:
-    - An essential pre-processing tool, specifically for textual analysis in machine learning model data.
-    - Provides a comprehensive breakdown of a text dataset, which aids in understanding both structural and vocabulary
-    complexity.
-    - Generates visualizations of correlations between chosen variables to further comprehend the text's structure and
-    complexity.
-
-    **Limitations**:
-    - Heavy reliance on the NLTK library, restricting its use to only the languages that NLTK supports.
-    - Limited customization capacity as the undesirable tokens and stop words are predefined.
-    - Lacks the ability to consider semantics or grammatical complexities, which could be crucial aspects in language
-    processing.
-    - Assumes that the document is well-structured (includes sentences and paragraphs); therefore, unstructured or
-    poorly formatted text may distort the results.
+
+    ### Strengths
+
+    - Essential for pre-processing text data in machine learning models.
+    - Provides a comprehensive breakdown of text data, aiding in understanding its complexity.
+    - Generates visualizations to help comprehend text structure and complexity.
+
+    ### Limitations
+
+    - Highly dependent on the NLTK library, limiting the test to supported languages.
+    - Limited customization for removing undesirable tokens and stop words.
+    - Does not consider semantic or grammatical complexities.
+    - Assumes well-structured documents, which may result in inaccuracies with poorly formatted text.
     """
 
     name = "text_description"

validmind/tests/data_validation/nlp/Toxicity.py

@@ -13,18 +13,41 @@ from validmind import tags, tasks
 @tasks("nlp")
 def Toxicity(dataset):
     """
-    Analyzes the toxicity of text data within a dataset using a pre-trained toxicity model.
+    Assesses the toxicity of text data within a dataset to visualize the distribution of toxicity scores.
 
-    This method loads a toxicity evaluation model and applies it to each text entry
-    in the specified column of the dataset's dataframe. It returns a KDE plot visualizing the distribution
-    of toxicity scores across the dataset.
+    ### Purpose
 
-    Args:
-        dataset (Dataset): A dataset object which must have a `df` attribute (a pandas DataFrame)
-            and a `text_column` attribute indicating the name of the column containing text.
+    The Toxicity test aims to evaluate the level of toxic content present in a text dataset by leveraging a pre-trained
+    toxicity model. It helps in identifying potentially harmful or offensive language that may negatively impact users
+    or stakeholders.
 
-    Returns:
-        matplotlib.figure.Figure: A KDE plot visualizing the distribution of toxicity scores.
+    ### Test Mechanism
+
+    This test uses a pre-trained toxicity evaluation model and applies it to each text entry in the specified column of
+    a dataset’s dataframe. The procedure involves:
+
+    - Loading a pre-trained toxicity model.
+    - Extracting the text from the specified column in the dataset.
+    - Computing toxicity scores for each text entry.
+    - Generating a KDE (Kernel Density Estimate) plot to visualize the distribution of these toxicity scores.
+
+    ### Signs of High Risk
+
+    - High concentration of high toxicity scores in the KDE plot.
+    - A significant proportion of text entries with toxicity scores above a predefined threshold.
+    - Wide distribution of toxicity scores, indicating inconsistency in content quality.
+
+    ### Strengths
+
+    - Provides a visual representation of toxicity distribution, making it easier to identify outliers.
+    - Uses a robust pre-trained model for toxicity evaluation.
+    - Can process large text datasets efficiently.
+
+    ### Limitations
+
+    - Depends on the accuracy and bias of the pre-trained toxicity model.
+    - Does not provide context-specific insights, which may be necessary for nuanced understanding.
+    - May not capture all forms of subtle or indirect toxic language.
     """
     toxicity = evaluate.load("toxicity")
     input_text = dataset.df[dataset.text_column]