quick-sentiments 0.1.9__tar.gz

quick_sentiments-0.1.9/PKG-INFO

@@ -0,0 +1,133 @@
+Metadata-Version: 2.4
+Name: quick-sentiments
+Version: 0.1.9
+Summary: Sentiment Analysis pipeline
+Author-email: Alabhya Dahal <alabhya.dahal@gmail.com>
+License: MIT License
+Project-URL: Homepage, https://github.com/AlabhyaMe/Sentiments-Analysis.git
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: numpy<2.0.0,>=1.23.0
+Requires-Dist: pandas>=1.3.0
+Requires-Dist: scikit-learn>=1.0.0
+Requires-Dist: polars<0.20.0,>=0.19.5
+Requires-Dist: gensim>4.0.0
+Requires-Dist: nltk==3.8.1
+Requires-Dist: xgboost==3.0.2
+Requires-Dist: scipy<2.0.0,>=1.9.0
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0; extra == "dev"
+Requires-Dist: black>=22.0; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Requires-Dist: build; extra == "dev"
+
+# 💬 Sentiment Analysis Pipeline
+
+This repository hosts an easy-to-use, ready-made **Sentiment Analysis pipeline** designed to get you started quickly with classifying text data. Everything you need, from data preprocessing to model training and prediction, is set up and configured.
+
+---
+
+## ✨ Features
+
+- **End-to-End Pipeline**: Go from raw text to sentiment predictions with minimal setup.
+- **Automated Preprocessing**: Includes robust text cleaning:
+  - Lemmatization
+  - Stop word removal
+  - Punctuation handling
+  - URL/emoji/HTML removal, etc.
+- **Multiple Text Representation Methods**:
+  - Bag-of-Words (BoW)
+  - Term Frequency (TF)
+  - TF-IDF (Term Frequency-Inverse Document Frequency)
+  - Word Embeddings (Word2Vec - pre-trained Google News 300-dim model)
+- **Multiple Machine Learning Algorithms**:
+  - Logistic Regression
+  - Random Forest
+  - XGBoost
+- **Hyperparameter Tuning Support**:
+  - All models are compatible with GridSearchCV.
+  - By default, models run with standard parameters for quick testing.
+  - Grid search options are built-in and ready to use if needed.
+- **Modular Design**: Each component is cleanly separated into its own module.
+- **Prediction on New Data**: Easily apply your trained model to new, unseen data.
+
+---
+
+## 🚀 Getting Started
+
+Follow these steps to get your sentiment analysis pipeline up and running:
+
+### 1. Prerequisites
+
+- **Git**: For cloning the repository.
+- **Python 3.8+** (Recommended: Anaconda for environment management)
+- **Anaconda/Miniconda**: Strongly recommended
+
+### 2. Clone the Repository
+
+```bash
+git clone https://github.com/AlabhyaMe/Sentimental-Analysis-.git
+cd Sentimental-Analysis-
+conda create -n sentiment_env python=3.9
+conda activate sentiment_env
+pip install -r requirements.txt
+```
+```
+This project is setup in the follwing pipeline
+├── Training Data/
+│   └── train.csv                # Your training file
+├── New Data/
+│   └── new_texts.csv            # Your new prediction file
+├── MLAlgo/
+│   ├── logistic_regression_model.py
+│   ├── random_forest_model.py
+│   └── xgboost_model.py
+├── Vect/
+│   ├── bag_of_words_vectorizer.py
+│   ├── tfidf_vectorizer.py
+│   └── word_embedding_vectorizer.py
+├── preprocessing.py             # Text cleaning functions
+├── sentiment_analysis_main.ipynb  # Full training + prediction notebook
+├── sentiment_prediction.ipynb     # Quick prediction-only notebook
+├── requirements.txt
+└── README.md
+
+
+```
+
+
+## 3. Prepare Your Data
+
+### 📌 Training Data
+
+Place your training CSV file in the `Training Data/` folder.
+
+- It must contain:
+  - A column named `Response` – for the raw input text.
+  - A column named `Sentiment` – for sentiment labels (e.g., `"Positive"`, `"Negative"`, `"Neutral"`).
+
+### 📌 New Data for Prediction
+
+Place your new prediction CSV file in the `New Data/` folder.
+
+- It must contain:
+  - A column named `RawTextColumn` (or another name you configure in the notebook).
+
+## 📚 Dataset Citation
+
+This project uses publicly available training data from:
+
+> Madhav Kumar Choudhary. *Sentiment Prediction on Movie Reviews*. Kaggle.  
+> [https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews](https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews)  
+> Accessed on: 2025- 07-15
+
+If you use this dataset in your own work, please cite the original creator as per Kaggle's [Terms of Use](https://www.kaggle.com/terms).
+

quick_sentiments-0.1.9/README.md

@@ -0,0 +1,102 @@
+# 💬 Sentiment Analysis Pipeline
+
+This repository hosts an easy-to-use, ready-made **Sentiment Analysis pipeline** designed to get you started quickly with classifying text data. Everything you need, from data preprocessing to model training and prediction, is set up and configured.
+
+---
+
+## ✨ Features
+
+- **End-to-End Pipeline**: Go from raw text to sentiment predictions with minimal setup.
+- **Automated Preprocessing**: Includes robust text cleaning:
+  - Lemmatization
+  - Stop word removal
+  - Punctuation handling
+  - URL/emoji/HTML removal, etc.
+- **Multiple Text Representation Methods**:
+  - Bag-of-Words (BoW)
+  - Term Frequency (TF)
+  - TF-IDF (Term Frequency-Inverse Document Frequency)
+  - Word Embeddings (Word2Vec - pre-trained Google News 300-dim model)
+- **Multiple Machine Learning Algorithms**:
+  - Logistic Regression
+  - Random Forest
+  - XGBoost
+- **Hyperparameter Tuning Support**:
+  - All models are compatible with GridSearchCV.
+  - By default, models run with standard parameters for quick testing.
+  - Grid search options are built-in and ready to use if needed.
+- **Modular Design**: Each component is cleanly separated into its own module.
+- **Prediction on New Data**: Easily apply your trained model to new, unseen data.
+
+---
+
+## 🚀 Getting Started
+
+Follow these steps to get your sentiment analysis pipeline up and running:
+
+### 1. Prerequisites
+
+- **Git**: For cloning the repository.
+- **Python 3.8+** (Recommended: Anaconda for environment management)
+- **Anaconda/Miniconda**: Strongly recommended
+
+### 2. Clone the Repository
+
+```bash
+git clone https://github.com/AlabhyaMe/Sentimental-Analysis-.git
+cd Sentimental-Analysis-
+conda create -n sentiment_env python=3.9
+conda activate sentiment_env
+pip install -r requirements.txt
+```
+```
+This project is setup in the follwing pipeline
+├── Training Data/
+│   └── train.csv                # Your training file
+├── New Data/
+│   └── new_texts.csv            # Your new prediction file
+├── MLAlgo/
+│   ├── logistic_regression_model.py
+│   ├── random_forest_model.py
+│   └── xgboost_model.py
+├── Vect/
+│   ├── bag_of_words_vectorizer.py
+│   ├── tfidf_vectorizer.py
+│   └── word_embedding_vectorizer.py
+├── preprocessing.py             # Text cleaning functions
+├── sentiment_analysis_main.ipynb  # Full training + prediction notebook
+├── sentiment_prediction.ipynb     # Quick prediction-only notebook
+├── requirements.txt
+└── README.md
+
+
+```
+
+
+## 3. Prepare Your Data
+
+### 📌 Training Data
+
+Place your training CSV file in the `Training Data/` folder.
+
+- It must contain:
+  - A column named `Response` – for the raw input text.
+  - A column named `Sentiment` – for sentiment labels (e.g., `"Positive"`, `"Negative"`, `"Neutral"`).
+
+### 📌 New Data for Prediction
+
+Place your new prediction CSV file in the `New Data/` folder.
+
+- It must contain:
+  - A column named `RawTextColumn` (or another name you configure in the notebook).
+
+## 📚 Dataset Citation
+
+This project uses publicly available training data from:
+
+> Madhav Kumar Choudhary. *Sentiment Prediction on Movie Reviews*. Kaggle.  
+> [https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews](https://www.kaggle.com/datasets/madhavkumarchoudhary/sentiment-prediction-on-movie-reviews)  
+> Accessed on: 2025- 07-15
+
+If you use this dataset in your own work, please cite the original creator as per Kaggle's [Terms of Use](https://www.kaggle.com/terms).
+

quick_sentiments-0.1.9/demo/sentiment_prediction.py

@@ -0,0 +1,67 @@
+! pip install .\dist\quick_sentiments-0.1.6-py3-none-any.whl
+#load the packages
+import polars as pl
+#make sure that this is the main file
+import sys
+import os
+project_root = os.getcwd()
+sys.path.insert(0, project_root)
+
+# here I have three python script I built to pre_process the data and running the pipeline
+# you can find the code in the tools/preprocess.py file
+# you can find  the code in the tools/pipeline.py file
+# the pre_process function is used to clean the text data, there are various options available, please check the tools/preprocess.py file for details
+# the run_pipeline function is used to run the sentimental analysis pipeline, it takes the training data and the vectorizer and machine learning methods as input, and returns the results
+
+# the run_pipeline function is used to run the sentimental analysis pipeline, it takes the training data and the vectorizer and machine learning methods as input, and returns the results
+from quick_sentiments import pre_process
+#this function will run the sentimental analysis in the training data and return the results
+from quick_sentiments import run_pipeline
+# this function will run the sentimental analysis in the new data and return the predictions
+from quick_sentiments import make_predictions
+
+# ENTER YOUR PATHS HERE FOR THE TRAINING DATA SET
+path1 = "training_data/train.csv" #give path to the training data
+df_train = pl.read_csv(path1, has_header=True, encoding="utf8")
+
+# ENTER YOUR PATHS HERE FOR THE TESTING DATA SET
+path2 = "new_data/test.csv" #give path to the test data
+df_test = pl.read_csv(path2, has_header=True, encoding="utf8")
+
+
+# ENTER THE COLUMN NAMES HERE
+response_column = "reviewText" # feel free to change the column name to your text column name
+sentiment_column = "sentiment" # feel free to change the column name to your label column name   
+
+df_train = df_train.with_columns(
+    pl.col(response_column).map_elements(lambda x: pre_process(x)).alias("processed")  #add inside the map_elements
+)
+
+dt= run_pipeline(
+    vectorizer_name="BOW", # BOW, tf, tfidf, wv
+    model_name="logit", # logit, rf, XGB .#XGB takes long time, can not recommend using it on normal case
+    df=df_train,
+    text_column_name="processed",  # this is the column name of the text data, 
+    sentiment_column_name = "sentiment",
+    perform_tuning = False # make this true if you want to perform hyperparameter tuning, it will take longer time and 
+                            # may run out of memory if the dataset is large,
+)
+
+new_data = df_test.with_columns(
+    pl.col(response_column).map_elements(lambda x: pre_process(x).alias("processed")  #add inside the map_elements
+))
+
+# MAKE PREDICTIONS ON THE NEW DATA
+sentiments_prediction= make_predictions(
+    new_data=new_data,
+    text_column_name="processed",
+    vectorizer=dt["vectorizer_object"],
+    best_model=dt["model_object"],
+    label_encoder=dt["label_encoder"],
+    prediction_column_name="sentiment_predictions"  # Optional custom name
+)
+
+# SAVE THE PREDICTIONS TO A CSV FILE
+sentiments_prediction.write_csv("new_data/sentiments_prediction.csv")
+
+# THE END

quick_sentiments-0.1.9/pyproject.toml

@@ -0,0 +1,50 @@
+# pyproject.toml 
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "quick-sentiments"
+version = "0.1.9" # new version
+authors = [
+  { name="Alabhya Dahal", email="alabhya.dahal@gmail.com" },
+]
+description = "Sentiment Analysis pipeline"
+readme = "README.md" # <--- Ensure this points to your README.md
+license = { text = "MIT License" }
+urls = { "Homepage" = "https://github.com/AlabhyaMe/Sentiments-Analysis.git" }
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+requires-python = ">=3.8"
+dependencies = [
+    "numpy>=1.23.0 ,<2.0.0",
+    "pandas>=1.3.0",
+    "scikit-learn>=1.0.0 ",
+    "polars>=0.19.5,<0.20.0", 
+    "gensim>4.0.0",
+    "nltk==3.8.1",
+    "xgboost==3.0.2",
+    "scipy>=1.9.0,<2.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=6.0",
+    "black>=22.0",
+    "twine",
+    "build",
+]
+
+[tool.setuptools.packages.find]
+where = ["."]
+
+[tool.setuptools.package-data]
+"quick_sentiments" = ["*.json", "*.pkl"]

quick_sentiments-0.1.9/quick_sentiments/__init__.py

@@ -0,0 +1,7 @@
+# quick_sentiments/__init__.py
+from .pipeline import run_pipeline       # Expose pipeline function
+from .predict import make_predictions # Expose prediction function
+from .preprocess import pre_process  # Expose preprocessing function
+
+__all__ = ['run_pipeline', 'make_predictions']  # Controls what's available in *
+

quick_sentiments-0.1.9/quick_sentiments/ml_algo/XGB.py

@@ -0,0 +1,91 @@
+from xgboost import XGBClassifier # Import XGBClassifier
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test, perform_tuning: bool = True): # <--- Added perform_tuning flag
+    """
+    Trains XGBoostClassifier model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (numerical, e.g., 0, 1, 2...).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained XGBoostClassifier model (either from GridSearchCV or simple fit).
+    """
+    print("   - Starting XGBoost training...")
+
+    # Determine objective and eval_metric based on number of unique classes
+    num_classes = len(np.unique(y_train))
+    
+    if num_classes == 2:
+        xgb_objective = 'binary:logistic'
+        xgb_eval_metric = 'logloss'
+        scoring_metric = 'f1_weighted'
+    else:
+        xgb_objective = 'multi:softmax'
+        xgb_eval_metric = 'mlogloss'
+        scoring_metric = 'f1_weighted' # Or 'accuracy'
+
+    # Base XGBClassifier model (used for both tuning and simple fit)
+    # verbosity=0 to suppress excessive output from XGBoost itself during GridSearchCV
+    xgb_model = XGBClassifier(
+        objective=xgb_objective,
+        eval_metric=xgb_eval_metric,
+        use_label_encoder=False, # Suppress warning for newer versions
+        random_state=42,
+        num_class=num_classes if num_classes > 2 else None,
+        verbosity=0 # Suppress XGBoost internal verbosity during grid search
+    )
+
+
+    if perform_tuning:
+        print("   - Performing GridSearchCV for hyperparameter tuning...")
+
+       
+        param_grid = {
+            'n_estimators': [100, 200],
+            'learning_rate': [0.05, 0.1, 0.2],
+            'max_depth': [3, 5, 7],
+            'subsample': [0.8, 1.0],
+            'colsample_bytree': [0.8, 1.0],
+        
+        
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=xgb_model, # Use the base xgb_model here
+            param_grid=param_grid,
+            cv=5,
+            scoring=scoring_metric,
+            n_jobs=-1,
+            verbose=1 # Print progress messages from GridSearchCV
+        )
+
+        # Fit GridSearchCV to the training data
+        grid_search.fit(X_train, y_train)
+
+        # Get the best model found by GridSearchCV
+        best_model = grid_search.best_estimator_
+
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score ({scoring_metric}): {grid_search.best_score_:.4f}")
+    else:
+        
+        print("   - Training XGBoost with default parameters (no hyperparameter tuning)...")
+        best_model = xgb_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    # Make predictions on the test set using the best model (tuned or default)
+    y_pred = best_model.predict(X_test)
+
+    # Return both the predictions and the best model object
+    return y_pred, best_model

quick_sentiments-0.1.9/quick_sentiments/ml_algo/logit.py

@@ -0,0 +1,60 @@
+# MLAlgo/logistic_regression_model.py
+
+from sklearn.linear_model import LogisticRegression
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test,perform_tuning = False):
+    """
+        Trains Logistic Regression model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (list or NumPy array).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained LogisticRegression model (either from GridSearchCV or simple fit).
+    """
+    lr_model = LogisticRegression(random_state=42) # Base model for training
+
+    if perform_tuning:
+        print("   - Starting Logistic Regression training with GridSearchCV for hyperparameter tuning...")
+
+        # Define the parameter grid to search (default grid, as no custom grid is passed here)
+        param_grid = {
+            'solver': ['liblinear', 'lbfgs'],
+            'C': [0.1, 1.0, 10.0],
+            'class_weight': [None, 'balanced'],
+            'max_iter': [500, 1000]
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=lr_model,
+            param_grid=param_grid,
+            cv=5,
+            scoring='f1_weighted',
+            n_jobs=-1,
+            verbose=1
+        )
+
+        grid_search.fit(X_train, y_train)
+        best_model = grid_search.best_estimator_
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score (F1-weighted): {grid_search.best_score_:.4f}")
+    else:
+        print("   - Training Logistic Regression with default parameters (no hyperparameter tuning)...")
+        best_model = lr_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    y_pred = best_model.predict(X_test)
+    print("Best model parameters:", best_model.get_params())
+
+    return y_pred, best_model

quick_sentiments-0.1.9/quick_sentiments/ml_algo/rf.py

@@ -0,0 +1,67 @@
+# MLAlgo/random_forest_model.py
+
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.model_selection import GridSearchCV # Import GridSearchCV
+from sklearn.metrics import classification_report # For evaluation metrics
+import numpy as np # For type hinting
+
+def train_and_predict(X_train, y_train, X_test, perform_tuning = False):
+    
+    """
+    Trains RandomForestClassifier model (with optional hyperparameter tuning) and predicts on test data.
+
+    Args:
+        X_train: training features (e.g., NumPy array or sparse matrix).
+        y_train: training labels (list or NumPy array).
+        X_test: test features (e.g., NumPy array or sparse matrix).
+        perform_tuning (bool): If True, performs GridSearchCV. If False, trains
+                               the model with default parameters. Defaults to True.
+
+    Returns:
+        y_pred: predicted labels for test set.
+        best_model: The best trained RandomForestClassifier model (either from GridSearchCV or simple fit).
+    """
+    rf_model = RandomForestClassifier(random_state=42) # Base model for training
+
+    if perform_tuning:
+        print("   - Starting Random Forest training with GridSearchCV for hyperparameter tuning...")
+
+        # Define the parameter grid to search for RandomForestClassifier (default grid)
+        param_grid = {
+            'n_estimators': [100, 200, 300],
+            'max_depth': [10, 20, None],
+            'min_samples_split': [2, 5],
+            'min_samples_leaf': [1, 2],
+            'class_weight': [None, 'balanced']
+        }
+        print("   - Using default parameter grid for tuning:", param_grid)
+
+        grid_search = GridSearchCV(
+            estimator=rf_model, # Use the base rf_model here
+            param_grid=param_grid,
+            cv=5,
+            scoring='f1_weighted',
+            n_jobs=-1,
+            verbose=1
+        )
+
+        # Fit GridSearchCV to the training data
+        grid_search.fit(X_train, y_train)
+
+        # Get the best model found by GridSearchCV
+        best_model = grid_search.best_estimator_
+
+        print("\n   - Best Hyperparameters found:")
+        print(grid_search.best_params_)
+        print(f"   - Best Cross-Validation Score (F1-weighted): {grid_search.best_score_:.4f}")
+    else:
+        print("   - Training Random Forest with default parameters (no hyperparameter tuning)...")
+        best_model = rf_model # Use the base model directly
+        best_model.fit(X_train, y_train) # Fit it on X_train, y_train
+        print("   - Model trained with default parameters.")
+
+    # Make predictions on the test set using the best model (tuned or default)
+    y_pred = best_model.predict(X_test)
+
+    # Return both the predictions and the best model object
+    return y_pred, best_model