autoforge-engine 0.1.3__tar.gz → 1.0.0__tar.gz

autoforge_engine-1.0.0/PKG-INFO

@@ -0,0 +1,425 @@
+Metadata-Version: 2.4
+Name: autoforge-engine
+Version: 1.0.0
+Summary: A transparent, local-first AutoML framework for automated ML pipeline discovery.
+Author: Aditya Kumar Singh
+Author-email: Aditya Kumar Singh <adityasingh45245@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/Adityasinghrajput01/ModelForge
+Project-URL: Repository, https://github.com/Adityasinghrajput01/ModelForge
+Project-URL: Issues, https://github.com/Adityasinghrajput01/ModelForge/issues
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26
+Requires-Dist: pandas>=2.1
+Requires-Dist: scikit-learn>=1.4
+Requires-Dist: rich>=13.7
+Requires-Dist: typer>=0.12
+Requires-Dist: pyyaml>=6.0
+Provides-Extra: boosting
+Requires-Dist: xgboost>=2.0; extra == "boosting"
+Requires-Dist: lightgbm>=4.0; extra == "boosting"
+Requires-Dist: catboost>=1.2; extra == "boosting"
+Provides-Extra: optimization
+Requires-Dist: optuna>=3.6; extra == "optimization"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Requires-Dist: black>=24.0; extra == "dev"
+Requires-Dist: mypy>=1.10; extra == "dev"
+Dynamic: license-file
+
+<div align="center">
+
+# 🔨 ModelForge
+
+**A transparent, local-first AutoML framework for Python**
+
+[![PyPI version](https://img.shields.io/pypi/v/autoforge-engine.svg)](https://pypi.org/project/autoforge-engine/)
+[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://pypi.org/project/autoforge-engine/)
+[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+[![Tests](https://img.shields.io/badge/tests-491%20passing-brightgreen.svg)](#testing--validation)
+[![GitHub](https://img.shields.io/badge/GitHub-ModelForge-181717?logo=github)](https://github.com/Adityasinghrajput01/ModelForge)
+
+*Give it a dataset and a target column. It handles the rest — transparently.*
+
+[Installation](#installation) •
+[Quick Start](#quick-start) •
+[Configuration](#configuration) •
+[Architecture](#architecture) •
+[Stress Test Results](#stress-test-results) •
+[Roadmap](#roadmap)
+
+</div>
+
+---
+
+## What is ModelForge?
+
+**ModelForge** is a local-first AutoML framework that automates the repetitive parts of a machine learning workflow — data profiling, quality auditing, preprocessing, pipeline construction, model evaluation, cross-validation, ranking, and reporting — while staying inspectable at every step.
+
+Unlike black-box AutoML tools, ModelForge is built around one core principle:
+
+> **You should always be able to see what it did and why.**
+
+Give it a dataset and a target column:
+
+```python
+import pandas as pd
+from modelforge.automl import AutoML
+
+df = pd.read_csv("dataset.csv")
+
+automl = AutoML()
+automl.fit(df, target="target_column")
+```
+
+And it walks through the full pipeline automatically:
+
+```
+Dataset → Loading → Profiling → Data Quality Audit → Target/Task Detection
+        → Column Intelligence → Feature Selection/Engineering → Preprocessing
+        → Pipeline Generation → Multi-Model Evaluation → Cross-Validation
+        → Metric Calculation → Model Ranking → Best Model Selection
+        → (Optional) Hyperparameter Optimization → Explainability
+        → Persistence/Artifacts → Experiment Tracking → Human-Readable Report
+```
+
+---
+
+## Installation
+
+```bash
+pip install autoforge-engine
+```
+
+Install a specific version:
+
+```bash
+pip install autoforge-engine==0.1.4
+```
+
+Verify the install:
+
+```bash
+pip show autoforge-engine
+python -c "import modelforge; print(modelforge.__file__)"
+```
+
+> **Note:** the PyPI distribution name is `autoforge-engine`, but the importable package is `modelforge`.
+
+---
+
+## Quick Start
+
+### Using the `AutoML` class
+
+```python
+import pandas as pd
+from modelforge.automl import AutoML
+
+df = pd.read_csv("autoforge_10000_test_dataset.csv")
+
+automl = AutoML()
+automl.fit(df, target="loan_default")
+```
+
+### Using the convenience function
+
+```python
+from modelforge.automl import automl
+
+result = automl(
+    data=df,
+    target="loan_default"
+)
+```
+
+### Command line
+
+```bash
+modelforge --help
+```
+
+---
+
+## Selecting and Exporting a Model
+
+ModelForge evaluates and ranks candidate models automatically. By default,
+`best_model` / `best_pipeline` hold the top-ranked result, and `save()` persists
+that automatically recommended pipeline.
+
+You can override the recommendation after `fit()` without re-running AutoML.
+The selected candidate is refitted on the full training dataset before it is
+used for prediction or export. The exported artifact is the complete
+scikit-learn pipeline (preprocessing + estimator). Metadata records how the
+model was selected.
+
+### Automatic behavior
+
+```python
+automl.fit(df, target="target")
+automl.save("best_model.joblib")
+```
+
+### Select by rank
+
+```python
+automl.fit(df, target="target")
+
+automl.select_model(rank=2)
+
+automl.export_model(
+    "selected_model.joblib"
+)
+```
+
+### Select by model name
+
+```python
+automl.select_model(
+    model="random_forest_classifier"
+)
+
+automl.export_model(
+    "random_forest_model.joblib"
+)
+```
+
+Notes:
+
+- `best_model` remains the automatically ranked best model.
+- `selected_model` / `selected_pipeline` hold the user override.
+- After `select_model()`, `predict()` and `predict_proba()` use the selected
+  pipeline. Before selection they continue to use the best pipeline.
+- `save()` always saves the automatically ranked best model.
+- `export_model()` saves the selected model when one was chosen; otherwise it
+  saves the best model.
+
+### CLI selection
+
+Export a ranked model without changing the default train workflow:
+
+```bash
+modelforge train \
+    --data data.csv \
+    --target target \
+    --model-rank 2 \
+    --output random_forest_model.joblib
+```
+
+Or select by registry model name:
+
+```bash
+modelforge train \
+    --data data.csv \
+    --target target \
+    --model random_forest_classifier \
+    --output random_forest_model.joblib
+```
+
+Existing `modelforge train` commands without `--model` / `--model-rank`
+continue to save the automatically recommended best model.
+
+## Configuration
+
+The `AutoML` constructor supports the following parameters:
+
+```python
+AutoML(
+    test_size=0.2,
+    cv=5,
+    random_state=42,
+    objective="balanced",
+    variance_threshold=None,
+    correlation_threshold=None,
+    enable_optimization=False,
+    optimization_models=3,
+    optimization_max_trials=10,
+    experiment_directory=".modelforge/experiments",
+    config=None
+)
+```
+
+| Parameter | Description |
+|---|---|
+| `test_size` | Holdout test fraction |
+| `cv` | Number of cross-validation folds |
+| `random_state` | Reproducibility seed |
+| `objective` | Model ranking objective (default: `"balanced"`) |
+| `variance_threshold` | Optional low-variance feature filtering |
+| `correlation_threshold` | Optional high-correlation feature filtering |
+| `enable_optimization` | Enables/disables hyperparameter optimization |
+| `optimization_models` | Number of models considered for optimization |
+| `optimization_max_trials` | Maximum optimization trials |
+| `experiment_directory` | Where experiment metadata/artifacts are stored |
+
+The automatic post-fit report can be suppressed with `print_report=False` (if supported by the installed `fit()` signature).
+
+---
+
+## What ModelForge Does
+
+### 🧹 Data Quality Auditing
+Automatically flags:
+- Missing values
+- Duplicate rows
+- Constant columns
+- Possible identifier columns
+- Other dataset-quality findings
+
+### 🎯 Target & Task Detection
+Given a target column, ModelForge determines whether the problem is **classification** or **regression**.
+
+### ⚙️ Preprocessing
+Builds a leakage-safe `scikit-learn` `Pipeline` / `ColumnTransformer`:
+
+| Data type | Steps |
+|---|---|
+| Numerical | `SimpleImputer` → `StandardScaler` |
+| Categorical | `SimpleImputer` → `OneHotEncoder` |
+
+Preprocessing is fit independently inside each cross-validation fold to prevent data leakage.
+
+### 🤖 Model Evaluation
+Evaluates multiple candidate models (verified against source for the current release), including — for classification tasks — Logistic Regression, Random Forest, Extra Trees, Gradient Boosting, K-Nearest Neighbors, Support Vector Classifier, and Decision Tree.
+
+### 🔁 Cross-Validation
+Uses `StratifiedKFold` (`n_splits=5`, `shuffle=True`, `random_state=42`) for classification tasks where class counts permit. Each fold clones the pipeline fresh, fits on the training split, and evaluates on the validation split.
+
+### 📊 Metrics
+
+**Classification:** Accuracy, Precision, Recall, F1 (`average="weighted"`, `zero_division=0`), ROC-AUC (via `predict_proba` or `decision_function`), Log Loss.
+
+**Regression:** R², Adjusted R², MAE, MSE, RMSE, MAPE.
+
+### 🏆 Model Ranking
+Candidate models are ranked according to the configured `objective` (default: `"balanced"`) rather than by a single metric alone.
+
+### 📄 Automatic Reporting
+After `fit()`, ModelForge generates a human-readable report covering dataset shape, target/task, data-quality findings, evaluated models and metrics, the selected model, pipeline steps, run/experiment IDs, and run duration.
+
+---
+
+## Architecture
+
+ModelForge is organized into focused modules, each with a single responsibility:
+
+| Module | Responsibility |
+|---|---|
+| `data_loader.py` | Dataset loading |
+| `profiler.py` | Dataset profiling |
+| `data_audit.py` | Data quality / auditing |
+| `target_selector.py` | Target and task detection |
+| `column_intelligence.py` | Column-level analysis |
+| `feature_engineering.py` | Feature engineering |
+| `feature_selection.py` | Feature selection |
+| `preprocessing.py` | Imputation, scaling, encoding |
+| `pipeline_generator.py` | sklearn pipeline construction |
+| `model_registry.py` | Candidate model catalog |
+| `model_screening.py` | Fast holdout evaluation |
+| `cross_validation.py` | K-fold cross-validation |
+| `evaluation.py` | Metric computation |
+| `ranking.py` | Model ranking logic |
+| `hyperparameter_optimization.py` | Optional HPO |
+| `explainability.py` | Model explainability |
+| `persistence.py` | Model save/load |
+| `artifact_manager.py` | Artifact management |
+| `experiment_tracker.py` | Experiment tracking |
+| `reproducibility.py` / `reproducibility_integration.py` | Reproducibility guarantees |
+| `prediction_validator.py` | Prediction-time validation |
+| `run_manager.py` | Run metadata |
+| `cli.py` | Typer-based CLI |
+| `automl.py` | Orchestration layer |
+| `config.py` | Configuration handling |
+
+> **Note on Model Screening vs. Cross-Validation:** these are distinct evaluation paths. `ModelScreeningEngine` performs a fast `train_test_split` (80/20, stratified where applicable) holdout evaluation, while `CrossValidationEngine` performs full K-fold evaluation. Holdout screening metrics and CV metrics should not be conflated.
+
+---
+
+## Stress Test Results
+
+ModelForge 0.1.4 was validated against a synthetic 10,020-row, 18-column dataset (target: `loan_default`) intentionally containing missing values, duplicates, a constant feature, and an identifier-like column.
+
+**Data quality detected:** 1,000 missing cells · 20 duplicate rows · 3 quality findings (all correctly identified)
+
+**Cross-validation accuracy:**
+
+| Model | CV Accuracy |
+|---|---|
+| Logistic Regression | 69.51% |
+| Gradient Boosting | 69.51% |
+| SVC | 69.12% |
+| Random Forest | 69.02% |
+| Extra Trees | 68.26% |
+| KNN | 64.52% |
+| Decision Tree | 58.59% |
+
+**Selected model:** Logistic Regression · **Run time:** ~33.25 seconds
+
+> These figures are a functional stress-test snapshot, not a claim that any single algorithm is universally best.
+
+### Independent Validation
+
+Results were cross-checked against raw `scikit-learn` implementations. Six of seven models matched almost exactly; Decision Tree showed a ~0.13 percentage-point difference (58.59% vs. 58.72%), currently logged as an open, low-priority investigation rather than a confirmed defect.
+
+---
+
+## Testing & Validation
+
+- ✅ 491 tests passing locally
+- ✅ GitHub Actions CI passing
+- ✅ Package built (`.tar.gz` + `.whl`) and validated with `twine check`
+- ✅ Published to PyPI: [`autoforge-engine`](https://pypi.org/project/autoforge-engine/0.1.4/)
+
+Test coverage includes unit, integration, CLI, preprocessing, model evaluation, cross-validation, ranking, persistence, experiment tracking, edge cases, and large mixed-type stress tests.
+
+---
+
+## Development Workflow
+
+```bash
+# Run tests
+pytest
+
+# Build the package
+python -m build
+
+# Validate the build
+python -m twine check dist/*
+
+# Upload to PyPI
+python -m twine upload dist/*
+```
+
+---
+
+## Roadmap
+
+- [ ] Investigate the minor Decision Tree CV discrepancy (`model_registry` / `pipeline_generator` / `preprocessing`)
+- [ ] Expand hyperparameter optimization coverage
+- [ ] Broaden regression model support
+- [ ] Deepen explainability outputs
+- [ ] Continued CLI and documentation improvements
+
+---
+
+## Philosophy
+
+ModelForge is built to demonstrate serious ML engineering practice — not to replace a data scientist's judgment. It automates repeatable, mechanical steps of an ML workflow (preprocessing, evaluation, cross-validation, ranking, reporting) while keeping every step reproducible and inspectable, so the framework never becomes a black box.
+
+---
+
+## Author
+
+**Aditya Kumar Singh**
+
+## License
+
+[MIT](LICENSE)
+
+## Links
+
+- 📦 PyPI: [autoforge-engine](https://pypi.org/project/autoforge-engine/)
+- 💻 GitHub: [ModelForge](https://github.com/Adityasinghrajput01/ModelForge)