quick-sentiments 0.2.6__tar.gz

quick_sentiments-0.2.6/PKG-INFO

@@ -0,0 +1,118 @@
+Metadata-Version: 2.4
+Name: quick-sentiments
+Version: 0.2.6
+Summary: Sentiment Analysis pipeline
+Author: Alabhya Dahal
+Author-email: Alabhya Dahal <alabhya.dahal@gmail.com>
+License: MIT License
+Project-URL: Homepage, https://github.com/AlabhyaMe/Sentiments-Analysis
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: numpy<2.0.0,>=1.23.0
+Requires-Dist: pandas>=1.3.0
+Requires-Dist: scikit-learn>=1.0.0
+Requires-Dist: polars<0.20.0,>=0.19.5
+Requires-Dist: gensim>4.0.0
+Requires-Dist: nltk==3.8.1
+Requires-Dist: xgboost==3.0.2
+Requires-Dist: scipy<2.0.0,>=1.9.0
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0; extra == "dev"
+Requires-Dist: black>=22.0; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Dynamic: author
+
+#  Quick Sentiments
+
+## Updates
+The package is now live! 
+
+```bash
+!pip install quick-sentiments
+```
+Alternatively, you can clone the git and install it locally.
+
+```bash
+git clone https://github.com/AlabhyaMe/quick_sentiments.git
+```
+Then run the command in the command prompt or notebook where git is cloned. Make sure you are in the main directory - quick_sentiments
+
+```
+pip install .\dist\quick_sentiments-0.2.4-py3-none-any.whl # please note, sometimes I might not have updated the version number to the  latest
+```
+
+This Python package is designed to streamline natural language processing (NLP) for sentiment analysis. It achieves this by combining various vectorization techniques with machine learning models. The package automates the often complex and time-consuming vectorization process, allowing users to skip the manual coding typically required for this step. Additionally, users can easily select their preferred machine learning models to conduct sentiment analysis.
+
+
+##  Features
+
+- **End-to-End Pipeline**: Go from raw text to sentiment predictions with minimal setup.
+- **Automated Preprocessing**: Includes robust text cleaning:
+  - Lemmatization
+  - Stop word removal
+  - Punctuation handling
+  - URL/emoji/HTML removal, etc.
+- **Multiple Text Representation Methods**:
+  - Bag-of-Words (BoW)
+  - Term Frequency (TF)
+  - TF-IDF (Term Frequency-Inverse Document Frequency)
+  - Word Embeddings (Word2Vec - pre-trained Google News 300-dim model)
+  - Glove Embedding (25,50,100 and 200)
+- **Multiple Machine Learning Algorithms**:
+  - Logistic Regression
+  - Random Forest
+  - XGBoost
+  - Neural Network
+- **Hyperparameter Tuning Support**:
+  - All models are compatible with GridSearchCV.
+  - By default, models run with standard parameters for quick testing.
+  - Grid search options are built-in and ready to use if needed.
+- **Modular Design**: Each component is cleanly separated into its own module.
+- **Prediction on New Data**: Easily apply your trained model to new, unseen data.
+
+---
+
+## 3. INSTRUCTIONS AND DEMO
+
+To help users get started with this package, I have documented comprehensive instructions and a demo workbook. Please begin by reviewing quick_sentiments.pdf for an introduction to the library's capabilities.
+
+Afterward, proceed to the Demo workbook, which contains ready-to-use examples. Please ensure that your file names and column labels are accurately set before proceeding with the instructions within the workbook. As an alternative, you may directly execute the Python script, provided your files and labels are correctly configured.
+
+###  Training Data
+
+Place your training CSV file in the `demo/training_data` folder.
+
+- It must contain:
+  - A column for  the raw input text. 
+  - A column for sentiments
+
+### New Data for Prediction
+
+Place your new prediction CSV file in the `new_data/` folder.
+
+- It must contain:
+  - A column named `RawTextColumn` (or another name you configure in the notebook).
+
+## 📚 Dataset Citation
+
+The demo uses publicly available training data from:
+
+> Madhav Kumar Choudhary. *Sentiment Prediction on Movie Reviews*. Kaggle.  
+> [https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews](https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews)  
+> Accessed on: 2025- 07-15
+
+If you use this dataset in your own work, please cite the original creator as per Kaggle's [Terms of Use](https://www.kaggle.com/terms).
+

quick_sentiments-0.2.6/README.md

@@ -0,0 +1,81 @@
+#  Quick Sentiments
+
+## Updates
+The package is now live! 
+
+```bash
+!pip install quick-sentiments
+```
+Alternatively, you can clone the git and install it locally.
+
+```bash
+git clone https://github.com/AlabhyaMe/quick_sentiments.git
+```
+Then run the command in the command prompt or notebook where git is cloned. Make sure you are in the main directory - quick_sentiments
+
+```
+pip install .\dist\quick_sentiments-0.2.4-py3-none-any.whl # please note, sometimes I might not have updated the version number to the  latest
+```
+
+This Python package is designed to streamline natural language processing (NLP) for sentiment analysis. It achieves this by combining various vectorization techniques with machine learning models. The package automates the often complex and time-consuming vectorization process, allowing users to skip the manual coding typically required for this step. Additionally, users can easily select their preferred machine learning models to conduct sentiment analysis.
+
+
+##  Features
+
+- **End-to-End Pipeline**: Go from raw text to sentiment predictions with minimal setup.
+- **Automated Preprocessing**: Includes robust text cleaning:
+  - Lemmatization
+  - Stop word removal
+  - Punctuation handling
+  - URL/emoji/HTML removal, etc.
+- **Multiple Text Representation Methods**:
+  - Bag-of-Words (BoW)
+  - Term Frequency (TF)
+  - TF-IDF (Term Frequency-Inverse Document Frequency)
+  - Word Embeddings (Word2Vec - pre-trained Google News 300-dim model)
+  - Glove Embedding (25,50,100 and 200)
+- **Multiple Machine Learning Algorithms**:
+  - Logistic Regression
+  - Random Forest
+  - XGBoost
+  - Neural Network
+- **Hyperparameter Tuning Support**:
+  - All models are compatible with GridSearchCV.
+  - By default, models run with standard parameters for quick testing.
+  - Grid search options are built-in and ready to use if needed.
+- **Modular Design**: Each component is cleanly separated into its own module.
+- **Prediction on New Data**: Easily apply your trained model to new, unseen data.
+
+---
+
+## 3. INSTRUCTIONS AND DEMO
+
+To help users get started with this package, I have documented comprehensive instructions and a demo workbook. Please begin by reviewing quick_sentiments.pdf for an introduction to the library's capabilities.
+
+Afterward, proceed to the Demo workbook, which contains ready-to-use examples. Please ensure that your file names and column labels are accurately set before proceeding with the instructions within the workbook. As an alternative, you may directly execute the Python script, provided your files and labels are correctly configured.
+
+###  Training Data
+
+Place your training CSV file in the `demo/training_data` folder.
+
+- It must contain:
+  - A column for  the raw input text. 
+  - A column for sentiments
+
+### New Data for Prediction
+
+Place your new prediction CSV file in the `new_data/` folder.
+
+- It must contain:
+  - A column named `RawTextColumn` (or another name you configure in the notebook).
+
+## 📚 Dataset Citation
+
+The demo uses publicly available training data from:
+
+> Madhav Kumar Choudhary. *Sentiment Prediction on Movie Reviews*. Kaggle.  
+> [https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews](https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews)  
+> Accessed on: 2025- 07-15
+
+If you use this dataset in your own work, please cite the original creator as per Kaggle's [Terms of Use](https://www.kaggle.com/terms).
+

quick_sentiments-0.2.6/pyproject.toml

@@ -0,0 +1,54 @@
+# pyproject.toml 
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "quick-sentiments"
+version = "0.2.6" # minor updates
+authors = [
+  { name="Alabhya Dahal", email="alabhya.dahal@gmail.com" },
+]
+description = "Sentiment Analysis pipeline"
+readme = "README.md" # <--- Ensure this points to your README.md
+license = { text = "MIT License" }
+urls = { "Homepage" = "https://github.com/AlabhyaMe/Sentiments-Analysis" }
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Topic :: Scientific/Engineering",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Topic :: Text Processing :: Linguistic"
+]
+requires-python = ">=3.8"
+dependencies = [
+    "numpy>=1.23.0 ,<2.0.0",
+    "pandas>=1.3.0",
+    "scikit-learn>=1.0.0 ",
+    "polars>=0.19.5,<0.20.0", 
+    "gensim>4.0.0",
+    "nltk==3.8.1",
+    "xgboost==3.0.2",
+    "scipy>=1.9.0,<2.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=6.0",
+    "black>=22.0",
+    "twine",
+    "build",
+]
+
+[tool.setuptools.packages.find]
+where = ["."]
+
+[tool.setuptools.package-data]
+"quick_sentiments" = ["*.json", "*.pkl"]

quick_sentiments-0.2.6/quick_sentiments/__init__.py

@@ -0,0 +1,8 @@
+# quick_sentiments/__init__.py
+from .pipeline import run_pipeline       # Expose pipeline function
+from .predict import make_predictions # Expose prediction function
+from .preprocess import pre_process  # Expose preprocessing function
+from .spacy_preprocess import pre_process_spacy
+
+__all__ = ['run_pipeline', 'make_predictions']  # Controls what's available in *
+

quick_sentiments-0.2.6/quick_sentiments/ml_algo/XGB.py

@@ -0,0 +1,91 @@
+from xgboost import XGBClassifier # Import XGBClassifier
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test, perform_tuning: bool = True): # <--- Added perform_tuning flag
+    """
+    Trains XGBoostClassifier model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (numerical, e.g., 0, 1, 2...).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained XGBoostClassifier model (either from GridSearchCV or simple fit).
+    """
+    print("   - Starting XGBoost training...")
+
+    # Determine objective and eval_metric based on number of unique classes
+    num_classes = len(np.unique(y_train))
+    
+    if num_classes == 2:
+        xgb_objective = 'binary:logistic'
+        xgb_eval_metric = 'logloss'
+        scoring_metric = 'f1_weighted'
+    else:
+        xgb_objective = 'multi:softmax'
+        xgb_eval_metric = 'mlogloss'
+        scoring_metric = 'f1_weighted' # Or 'accuracy'
+
+    # Base XGBClassifier model (used for both tuning and simple fit)
+    # verbosity=0 to suppress excessive output from XGBoost itself during GridSearchCV
+    xgb_model = XGBClassifier(
+        objective=xgb_objective,
+        eval_metric=xgb_eval_metric,
+        use_label_encoder=False, # Suppress warning for newer versions
+        random_state=42,
+        num_class=num_classes if num_classes > 2 else None,
+        verbosity=0 # Suppress XGBoost internal verbosity during grid search
+    )
+
+
+    if perform_tuning:
+        print("   - Performing GridSearchCV for hyperparameter tuning...")
+
+       
+        param_grid = {
+            'n_estimators': [100, 200],
+            'learning_rate': [0.05, 0.1, 0.2],
+            'max_depth': [3, 5, 7],
+            'subsample': [0.8, 1.0],
+            'colsample_bytree': [0.8, 1.0],
+        
+        
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=xgb_model, # Use the base xgb_model here
+            param_grid=param_grid,
+            cv=5,
+            scoring=scoring_metric,
+            n_jobs=-1,
+            verbose=1 # Print progress messages from GridSearchCV
+        )
+
+        # Fit GridSearchCV to the training data
+        grid_search.fit(X_train, y_train)
+
+        # Get the best model found by GridSearchCV
+        best_model = grid_search.best_estimator_
+
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score ({scoring_metric}): {grid_search.best_score_:.4f}")
+    else:
+        
+        print("   - Training XGBoost with default parameters (no hyperparameter tuning)...")
+        best_model = xgb_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    # Make predictions on the test set using the best model (tuned or default)
+    y_pred = best_model.predict(X_test)
+
+    # Return both the predictions and the best model object
+    return y_pred, best_model

quick_sentiments-0.2.6/quick_sentiments/ml_algo/logit.py

@@ -0,0 +1,60 @@
+# MLAlgo/logistic_regression_model.py
+
+from sklearn.linear_model import LogisticRegression
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test,perform_tuning = False):
+    """
+        Trains Logistic Regression model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (list or NumPy array).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained LogisticRegression model (either from GridSearchCV or simple fit).
+    """
+    lr_model = LogisticRegression(random_state=42) # Base model for training
+
+    if perform_tuning:
+        print("   - Starting Logistic Regression training with GridSearchCV for hyperparameter tuning...")
+
+        # Define the parameter grid to search (default grid, as no custom grid is passed here)
+        param_grid = {
+            'solver': ['liblinear', 'lbfgs'],
+            'C': [0.1, 1.0, 10.0],
+            'class_weight': [None, 'balanced'],
+            'max_iter': [500, 1000]
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=lr_model,
+            param_grid=param_grid,
+            cv=5,
+            scoring='f1_weighted',
+            n_jobs=-1,
+            verbose=1
+        )
+
+        grid_search.fit(X_train, y_train)
+        best_model = grid_search.best_estimator_
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score (F1-weighted): {grid_search.best_score_:.4f}")
+    else:
+        print("   - Training Logistic Regression with default parameters (no hyperparameter tuning)...")
+        best_model = lr_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    y_pred = best_model.predict(X_test)
+    print("Best model parameters:", best_model.get_params())
+
+    return y_pred, best_model

quick_sentiments-0.2.6/quick_sentiments/ml_algo/nn.py

@@ -0,0 +1,62 @@
+from sklearn.neural_network import MLPClassifier
+from sklearn.model_selection import GridSearchCV
+from sklearn.metrics import classification_report
+import numpy as np 
+
+def train_and_predict(X_train, y_train, X_test, perform_tuning=False):
+    """
+    Trains a Multi-layer Perceptron (MLP) Classifier model (Neural Network)
+    (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (list or NumPy array).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to False.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained MLPClassifier model (either from GridSearchCV or simple fit).
+    """
+    # Base model for training, with a max_iter for convergence
+    mlp_model = MLPClassifier(random_state=42, max_iter=1000)
+
+    if perform_tuning:
+        print("   - Starting MLPClassifier training with GridSearchCV for hyperparameter tuning...")
+
+        # Define the parameter grid to search
+        param_grid = {
+            'hidden_layer_sizes': [(50,), (100,), (50, 50)],
+            'activation': ['relu', 'tanh'],
+            'solver': ['adam', 'sgd'],
+            'alpha': [0.0001, 0.001, 0.01]
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        # Initialize GridSearchCV
+        grid_search = GridSearchCV(
+            estimator=mlp_model,
+            param_grid=param_grid,
+            cv=3,  # Using 3-fold cross-validation for speed
+            scoring='f1_weighted',
+            n_jobs=-1,
+            verbose=1
+        )
+
+        grid_search.fit(X_train, y_train)
+        best_model = grid_search.best_estimator_
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score (F1-weighted): {grid_search.best_score_:.4f}")
+    else:
+        print("   - Training MLPClassifier with default parameters (no hyperparameter tuning)...")
+        best_model = mlp_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    y_pred = best_model.predict(X_test)
+    print("Best model parameters:", best_model.get_params())
+
+    return y_pred, best_model
+

quick_sentiments-0.2.6/quick_sentiments/ml_algo/rf.py

@@ -0,0 +1,67 @@
+# MLAlgo/random_forest_model.py
+
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test, perform_tuning = False):
+    
+    """
+    Trains RandomForestClassifier model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (list or NumPy array).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained RandomForestClassifier model (either from GridSearchCV or simple fit).
+    """
+    rf_model = RandomForestClassifier(random_state=42) # Base model for training
+
+    if perform_tuning:
+        print("   - Starting Random Forest training with GridSearchCV for hyperparameter tuning...")
+
+        # Define the parameter grid to search for RandomForestClassifier (default grid)
+        param_grid = {
+            'n_estimators': [100, 200, 300],
+            'max_depth': [10, 20, None],
+            'min_samples_split': [2, 5],
+            'min_samples_leaf': [1, 2],
+            'class_weight': [None, 'balanced']
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=rf_model, # Use the base rf_model here
+            param_grid=param_grid,
+            cv=5,
+            scoring='f1_weighted',
+            n_jobs=-1,
+            verbose=1
+        )
+
+        # Fit GridSearchCV to the training data
+        grid_search.fit(X_train, y_train)
+
+        # Get the best model found by GridSearchCV
+        best_model = grid_search.best_estimator_
+
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score (F1-weighted): {grid_search.best_score_:.4f}")
+    else:
+        print("   - Training Random Forest with default parameters (no hyperparameter tuning)...")
+        best_model = rf_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    # Make predictions on the test set using the best model (tuned or default)
+    y_pred = best_model.predict(X_test)
+
+    # Return both the predictions and the best model object
+    return y_pred, best_model

quick_sentiments-0.2.6/quick_sentiments/pipeline.py

@@ -0,0 +1,130 @@
+# POSITIVELY DO NOT CHANGE (This comment is for the user, I am making necessary fixes based on our conversation)
+
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import classification_report, accuracy_score
+from collections import Counter
+from sklearn.preprocessing import LabelEncoder
+import polars as pl
+import importlib
+import numpy as np
+import pandas as pd
+from typing import Union
+
+def run_pipeline(
+    vectorizer_name: str,
+    model_name: str,
+    df: Union[pl.DataFrame, pd.DataFrame],
+    text_column_name: str,
+    sentiment_column_name: str,
+    perform_tuning: bool = False
+):
+    """
+    Runs the full pipeline:
+      - vectorize
+      - train model
+      - evaluate
+
+    Args:
+        vectorizer_name (str): Name of the vectorizer (e.g., 'tfidf', 'word_embedding').
+        model_name (str): Name of the ML model (e.g., 'logistic_regression', 'random_forest').
+        df (pl.DataFrame): Your Polars DataFrame containing the text and sentiment columns.
+        text_column_name (str): The name of the column in `df` that contains the processed text.
+        sentiment_column_name (str): The name of the column in `df` that contains the sentiment labels.
+
+    Returns:
+        dict: A dictionary containing the trained model, fitted vectorizer, label encoder, and evaluation results.
+    """
+    print(f"--- Running Pipeline for {vectorizer_name.replace('_', ' ').title()} + {model_name.replace('_', ' ').title()} ---")
+
+    # Import vectorizer from vect folder
+    try:
+        vec_module = importlib.import_module(f"quick_sentiments.vect.{vectorizer_name}")
+        vectorize_train = getattr(vec_module, "vectorize_train")
+        vectorize_test = getattr(vec_module, "vectorize_test")
+    except (ImportError, AttributeError) as e:
+        print(f"Error loading vectorizer module/function: {e}")
+        return None
+
+    # Import ML model from ml_algo folder
+    try:
+        model_module = importlib.import_module(f"quick_sentiments.ml_algo.{model_name}")
+        train_and_predict_function = getattr(model_module, "train_and_predict")
+    except (ImportError, AttributeError) as e:
+        print(f"Error loading ML model module/function: {e}")
+        return None
+
+    """
+    Modified to handle both Polars and pandas DataFrames.
+    """
+    # Convert to Polars if input is pandas
+    if isinstance(df, pd.DataFrame):
+        df = pl.from_pandas(df)
+    elif not isinstance(df, pl.DataFrame):
+        raise TypeError(f"Expected Polars or pandas DataFrame, got {type(df)}")
+    
+    # Polars DataFrame handling
+    X_text = df[text_column_name].to_list()
+    y_raw = df[sentiment_column_name].to_list()
+    
+    # --- NEW: Check for and drop None values in X_text and y_raw ---
+    initial_data_len = len(X_text)
+    
+    # Filter out pairs where either X_text element or y_raw element is None
+    # Use zip to iterate over both lists simultaneously and filter
+    filtered_data = [(x, y_val) for x, y_val in zip(X_text, y_raw) if x is not None and y_val is not None]
+    
+    # Unzip the filtered data back into X_text and y_raw
+    if filtered_data: # Check if filtered_data is not empty to avoid unpacking error
+        X_text, y_raw = zip(*filtered_data)
+        X_text = list(X_text) # Convert back to list
+        y_raw = list(y_raw)   # Convert back to list
+    else:
+        # Handle case where all data might be None
+        print("WARNING: All data rows contained missing values after initial extraction. Cannot proceed with training.")
+        return None
+
+    dropped_rows_count = initial_data_len - len(X_text)
+    if dropped_rows_count > 0:
+        print(f"WARNING: Dropped {dropped_rows_count} rows due to missing values (None) in '{text_column_name}' or '{sentiment_column_name}' columns. Original rows: {initial_data_len}, Rows after dropping: {len(X_text)}")
+    else:
+        print("No missing values (None) found in text or sentiment columns. Proceeding with all rows.")
+    # ------------------------------------------------------------------
+
+    # Label Encoding for y_raw
+    label_encoder = LabelEncoder()
+    y = label_encoder.fit_transform(y_raw)
+    print(f"Labels encoded: Original -> {label_encoder.classes_}, Encoded -> {np.unique(y)}")
+
+    # Split data Before vectorization
+    print("1. Splitting data into train/test...")
+    X_train, X_test, y_train, y_test = train_test_split(
+        X_text, y, test_size=0.2, random_state=42, stratify=y
+    )
+
+    # Vectorize the dataset (X)
+    print("2. Vectorizing  dataset (X)...")
+    X_train_vectorized, fitted_vectorizer_object = vectorize_train(X_train)
+    X_test_vectorized = vectorize_test(X_test, fitted_vectorizer_object)
+
+    # Train + predict
+    print("3. Training and predicting...")
+    y_pred, trained_model_object = train_and_predict_function(X_train_vectorized, y_train, X_test_vectorized, perform_tuning=perform_tuning)
+
+    # Evaluate
+    print("4. Evaluating model...")
+    print("\nClassification Report:")
+    print(classification_report(y_test, y_pred, target_names=label_encoder.classes_))
+    print("True labels distribution:", Counter(y_test))
+    print("Predicted labels distribution:", Counter(y_pred))
+
+    # Return results including all necessary objects for future predictions
+    return {
+        "model_object": trained_model_object,
+        "vectorizer_name": vectorizer_name,
+        "vectorizer_object": fitted_vectorizer_object,
+        "label_encoder": label_encoder,
+        "y_test": y_test,
+        "y_pred": y_pred,
+        "accuracy": accuracy_score(y_test, y_pred),
+        "report": classification_report(y_test, y_pred, output_dict=True, target_names=label_encoder.classes_)
+    }