model-eval-toolkit 0.1.0__tar.gz

model_eval_toolkit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Raahul Krishna Durairaju
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

model_eval_toolkit-0.1.0/PKG-INFO

@@ -0,0 +1,339 @@
+Metadata-Version: 2.4
+Name: model-eval-toolkit
+Version: 0.1.0
+Summary: Unified ML evaluation framework for classification, regression, clustering, time series, NLP, CV, and recommendation systems.
+Author-email: Raahul Krishna Durairaju <rahulkrish28@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2026 Raahul Krishna Durairaju
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/RAAHUL-tech/model-eval-toolkit
+Project-URL: Documentation, https://github.com/RAAHUL-tech/model-eval-toolkit/tree/main/docs
+Project-URL: Repository, https://github.com/RAAHUL-tech/model-eval-toolkit
+Project-URL: Issues, https://github.com/RAAHUL-tech/model-eval-toolkit/issues
+Keywords: machine-learning,evaluation,metrics,nlp,computer-vision,time-series,ranking
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.23
+Requires-Dist: pandas>=1.5
+Requires-Dist: scikit-learn>=1.2
+Requires-Dist: matplotlib>=3.7
+Requires-Dist: seaborn>=0.12
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Requires-Dist: coverage[toml]>=7; extra == "test"
+Provides-Extra: pdf
+Requires-Dist: reportlab>=4; extra == "pdf"
+Provides-Extra: nlp
+Provides-Extra: vision
+Dynamic: license-file
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/RAAHUL-tech/model-eval-toolkit/main/docs/images/evalreport-logo.png" alt="Model Eval Toolkit" width="260">
+</p>
+
+# Model Eval Toolkit
+
+**Unified ML evaluation reports** for Python: metrics, plots, auto-insights, and export to **HTML**, **JSON**, **Markdown**, or **PDF**.
+
+Model Eval Toolkit provides a single, task-aware evaluation layer to benchmark model quality consistently across ML domains.
+
+Import from the **`evalreport`** package:
+
+```python
+from evalreport import (
+    generate_report,
+    ClassificationReport,
+    RegressionReport,
+    ClusteringReport,
+    TimeSeriesReport,
+    TextClassificationReport,
+    TextGenerationReport,
+    SegmentationReport,
+    DetectionReport,
+    RankingReport,
+    __version__,
+)
+```
+
+> **Current supported tasks (v0.1):**
+> classification (binary & multiclass), regression, clustering, time series/forecasting,
+> NLP (text classification + text generation), CV (segmentation + detection), and **recommendation / ranking**.
+> The roadmap includes multilabel and richer recsys (e.g. session-based, implicit feedback models).
+
+---
+
+## Install
+
+```bash
+pip install model-eval-toolkit
+```
+
+**PDF export** needs ReportLab:
+
+```bash
+pip install "model-eval-toolkit[pdf]"
+# or
+pip install reportlab
+```
+
+**Requirements:** Python ≥ 3.9, NumPy, pandas, scikit-learn, Matplotlib, Seaborn.
+
+Optional task extras (currently dependency-light for NLP/CV):
+
+```bash
+pip install "model-eval-toolkit[nlp]"
+pip install "model-eval-toolkit[vision]"
+```
+
+---
+
+## Quick start
+
+### `generate_report` (recommended)
+
+```python
+from evalreport import generate_report
+
+summary = generate_report(
+    task="classification",  # or "regression", or "auto"
+    y_true=[0, 1, 0, 1, 1],
+    y_pred=[0, 1, 1, 1, 1],
+    y_prob=[0.1, 0.9, 0.8, 0.7, 0.6],  # optional; enables log loss, ROC/PR (binary)
+    output_path="my_reports/model_report.html",
+    format="html",
+)
+
+print(summary["metrics"]["accuracy"])
+```
+
+NLP + CV examples:
+
+```python
+from evalreport import generate_report
+
+# Text generation
+generate_report(
+    task="text_generation",
+    y_true=["the cat sat on the mat"],
+    y_pred=["the cat sat on mat"],
+    output_path="reports/text_generation.html",
+)
+
+# Image segmentation (binary masks)
+generate_report(
+    task="segmentation",
+    y_true=[[[0, 0], [1, 1]]],
+    y_pred=[[[0, 1], [1, 1]]],
+    output_path="reports/segmentation.html",
+)
+
+# Object detection (per-image list of box dicts)
+generate_report(
+    task="detection",
+    y_true=[[{\"bbox\": [0, 0, 10, 10], \"label\": \"obj\"}]],
+    y_pred=[[{\"bbox\": [1, 1, 9, 9], \"label\": \"obj\", \"score\": 0.9}]],
+    output_path="reports/detection.html",
+)
+
+# Recommendation / ranking (one list per user)
+generate_report(
+    task="recommendation",  # or "ranking", "recommender"
+    y_true=[[10, 20], [30]],           # relevant item IDs per user
+    y_pred=[[10, 99, 20, 5], [7, 30]],  # ranked recommendations per user (best first)
+    k_values=(1, 5, 10),               # optional cutoffs for P@K, R@K, NDCG@K, Hit@K
+    output_path="reports/recommendation.html",
+)
+```
+
+- **`task="auto"`** — float targets → regression; integer/string labels → classification.
+- If you **omit `output_path`**, the report is written under **`reports/`** (created if needed), e.g. `reports/classification_report.html` or `reports/regression_report.json` when `format="json"`.
+- **Plots** are saved under **`<report_directory>/evalreport_plots/`** (same folder as your HTML/JSON/PDF file’s parent). So custom `output_path="my_reports/x.html"` → plots in `my_reports/evalreport_plots/`.
+
+### Task-specific API
+
+Useful when you want full control (e.g. set `output_dir` before `run_all()` so plots land next to a chosen folder):
+
+```python
+from pathlib import Path
+from evalreport import ClassificationReport, RegressionReport, RankingReport
+
+# Classification (binary or multiclass)
+cls = ClassificationReport(
+    y_true=[0, 1, 2, 0],
+    y_pred=[0, 2, 2, 0],
+    # y_prob: (n_samples, n_classes) for multiclass log loss / AUC
+    labels=[0, 1, 2],  # optional fixed class order for confusion matrix
+)
+cls.output_dir = Path("reports")  # optional; default for plots if set before run_all()
+cls.run_all()
+cls.save("reports/classification_report.html", format="html")
+cls.save("reports/classification_report.json", format="json")
+
+# Regression
+reg = RegressionReport(y_true=[1.0, 2.0, 3.0], y_pred=[1.1, 1.9, 3.2])
+reg.output_dir = Path("reports")
+reg.run_all()
+reg.save("reports/regression_report.pdf", format="pdf")  # needs reportlab
+
+# Recommendation / ranking
+rank = RankingReport(
+    relevant=[[1, 2], [3]],
+    ranked=[[1, 4, 5], [3, 1, 2]],
+    k_values=(1, 5, 10),
+)
+rank.output_dir = Path("reports")
+rank.run_all()
+rank.save("reports/ranking_report.html", format="html")
+```
+
+---
+
+## What each task includes
+
+### Classification
+
+| Area | Details |
+|------|--------|
+| **Metrics** | Accuracy; precision / recall / F1 (micro, macro, weighted); MCC; Cohen’s κ; log loss (with probs); ROC-AUC / PR-AUC when applicable; confusion matrix (table). |
+| **Plots** | Confusion matrix heatmap; **binary** ROC & PR curves when `y_prob` is provided. |
+| **Insights** | Class imbalance hint; most common misclassification pair. |
+| **HTML** | Styled layout: each metric with a short explanation, insights, and embedded plot images. |
+
+**Probabilities**
+
+- Binary: `y_prob` as length-`n` scores for the positive class, or shape `(n, 2)`.
+- Multiclass: `(n_samples, n_classes)` for log loss / multiclass AUC where supported.
+
+### Regression
+
+| Area | Details |
+|------|--------|
+| **Metrics** | MAE, MSE, RMSE, R², median absolute error, MAPE (where defined), mean error (bias). |
+| **Plots** | Residuals vs predicted, predicted vs actual, residual histogram. |
+| **Insights** | Over/under-prediction bias; heavy-tail error hint. |
+| **HTML** | Same rich layout as classification. |
+
+### Clustering
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `X` (feature matrix) and `labels` (cluster assignments) |
+| **Metrics** | Silhouette score, Davies–Bouldin index, Calinski–Harabasz score, cluster sizes |
+| **Plots** | Cluster scatter (PCA) and cluster size distribution |
+| **Insights** | Separability + imbalance hints |
+| **HTML** | Styled metrics/insights plus embedded plot images |
+
+### Time Series / Forecasting
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `y_true`, `y_pred`, and `timestamps` (same length) |
+| **Metrics** | MAE, MSE, RMSE, MAPE, SMAPE, mean forecast error, rolling RMSE summary |
+| **Plots** | Actual vs forecast, residuals over time, rolling RMSE over time |
+| **Insights** | Systematic bias and drift/stability hints via rolling RMSE |
+| **HTML** | Styled metrics/insights plus embedded plot images |
+
+### Recommendation / Ranking
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `relevant`: ground-truth relevant **item IDs** per user (or query). `ranked`: **ordered** recommended lists per user (same length as `relevant`). |
+| **Metrics** | **MAP** (binary relevance), **Precision@K**, **Recall@K**, **NDCG@K**, **Hit Rate@K** for each K in `k_values` (default `(1, 5, 10)`). |
+| **Plots** | Precision@K curve; mean **cumulative gain** vs rank cutoff. |
+| **Insights** | Drop in precision at larger K; long-tail spread in \#relevant per user; low-MAP hint. |
+| **`generate_report`** | `task="recommendation"` / `"ranking"` / `"recommender"` with `y_true=relevant`, `y_pred=ranked`. |
+
+---
+
+## Output formats
+
+| Format | How | Notes |
+|--------|-----|--------|
+| **HTML** | `format="html"` or `.html` | Metrics + descriptions + insights + plot images. |
+| **JSON** | `format="json"` or `.json` | `metrics`, `insights`, `plots` (paths to PNGs). |
+| **Markdown** | `format="markdown"` or `.md` | Metrics and insights (no embedded images). |
+| **PDF** | `format="pdf"` or `.pdf` | Text summary (metrics + descriptions + insights); install `reportlab`. |
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/RAAHUL-tech/model-eval-toolkit.git
+cd model-eval-toolkit
+python -m venv .venv
+source .venv/bin/activate  # Windows: .venv\Scripts\activate
+pip install -e ".[test]"
+pytest -q
+# optional coverage
+pytest --cov=evalreport --cov-report=term-missing
+```
+
+Build and check the package:
+
+```bash
+pip install build twine
+python -m build
+twine check dist/*
+```
+
+### CI and PyPI releases
+
+GitHub Actions (`.github/workflows/ci.yml`):
+
+- **Pull requests** → runs **tests** only (Python 3.9–3.11).
+- **Push to `main`** (including when a PR is merged) → runs **tests**, then **publishes** to [PyPI](https://pypi.org/project/model-eval-toolkit/) if tests pass.
+
+**One-time setup**
+
+1. On [pypi.org](https://pypi.org/manage/account/token/), create an **API token** scoped to this project (or your whole account for a first publish).
+2. In the GitHub repo: **Settings → Secrets and variables → Actions → New repository secret**
+   - Name: `PYPI_API_TOKEN`
+   - Value: the token (often starts with `pypi-`).
+
+**Before each release**
+
+- Bump `version` in `pyproject.toml`. PyPI rejects re-uploading the same version.
+
+Optional: use [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (OIDC) and drop the token; the workflow already requests `id-token: write` for that path.
+
+---
+
+## Roadmap
+
+Additional task types (clustering, time series, ranking, NLP, CV) and a plugin-style API are planned. Issues and PRs welcome on [GitHub](https://github.com/RAAHUL-tech/model-eval-toolkit).
+
+---
+
+## License
+
+See [LICENSE](LICENSE).

model_eval_toolkit-0.1.0/README.md

@@ -0,0 +1,281 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/RAAHUL-tech/model-eval-toolkit/main/docs/images/evalreport-logo.png" alt="Model Eval Toolkit" width="260">
+</p>
+
+# Model Eval Toolkit
+
+**Unified ML evaluation reports** for Python: metrics, plots, auto-insights, and export to **HTML**, **JSON**, **Markdown**, or **PDF**.
+
+Model Eval Toolkit provides a single, task-aware evaluation layer to benchmark model quality consistently across ML domains.
+
+Import from the **`evalreport`** package:
+
+```python
+from evalreport import (
+    generate_report,
+    ClassificationReport,
+    RegressionReport,
+    ClusteringReport,
+    TimeSeriesReport,
+    TextClassificationReport,
+    TextGenerationReport,
+    SegmentationReport,
+    DetectionReport,
+    RankingReport,
+    __version__,
+)
+```
+
+> **Current supported tasks (v0.1):**
+> classification (binary & multiclass), regression, clustering, time series/forecasting,
+> NLP (text classification + text generation), CV (segmentation + detection), and **recommendation / ranking**.
+> The roadmap includes multilabel and richer recsys (e.g. session-based, implicit feedback models).
+
+---
+
+## Install
+
+```bash
+pip install model-eval-toolkit
+```
+
+**PDF export** needs ReportLab:
+
+```bash
+pip install "model-eval-toolkit[pdf]"
+# or
+pip install reportlab
+```
+
+**Requirements:** Python ≥ 3.9, NumPy, pandas, scikit-learn, Matplotlib, Seaborn.
+
+Optional task extras (currently dependency-light for NLP/CV):
+
+```bash
+pip install "model-eval-toolkit[nlp]"
+pip install "model-eval-toolkit[vision]"
+```
+
+---
+
+## Quick start
+
+### `generate_report` (recommended)
+
+```python
+from evalreport import generate_report
+
+summary = generate_report(
+    task="classification",  # or "regression", or "auto"
+    y_true=[0, 1, 0, 1, 1],
+    y_pred=[0, 1, 1, 1, 1],
+    y_prob=[0.1, 0.9, 0.8, 0.7, 0.6],  # optional; enables log loss, ROC/PR (binary)
+    output_path="my_reports/model_report.html",
+    format="html",
+)
+
+print(summary["metrics"]["accuracy"])
+```
+
+NLP + CV examples:
+
+```python
+from evalreport import generate_report
+
+# Text generation
+generate_report(
+    task="text_generation",
+    y_true=["the cat sat on the mat"],
+    y_pred=["the cat sat on mat"],
+    output_path="reports/text_generation.html",
+)
+
+# Image segmentation (binary masks)
+generate_report(
+    task="segmentation",
+    y_true=[[[0, 0], [1, 1]]],
+    y_pred=[[[0, 1], [1, 1]]],
+    output_path="reports/segmentation.html",
+)
+
+# Object detection (per-image list of box dicts)
+generate_report(
+    task="detection",
+    y_true=[[{\"bbox\": [0, 0, 10, 10], \"label\": \"obj\"}]],
+    y_pred=[[{\"bbox\": [1, 1, 9, 9], \"label\": \"obj\", \"score\": 0.9}]],
+    output_path="reports/detection.html",
+)
+
+# Recommendation / ranking (one list per user)
+generate_report(
+    task="recommendation",  # or "ranking", "recommender"
+    y_true=[[10, 20], [30]],           # relevant item IDs per user
+    y_pred=[[10, 99, 20, 5], [7, 30]],  # ranked recommendations per user (best first)
+    k_values=(1, 5, 10),               # optional cutoffs for P@K, R@K, NDCG@K, Hit@K
+    output_path="reports/recommendation.html",
+)
+```
+
+- **`task="auto"`** — float targets → regression; integer/string labels → classification.
+- If you **omit `output_path`**, the report is written under **`reports/`** (created if needed), e.g. `reports/classification_report.html` or `reports/regression_report.json` when `format="json"`.
+- **Plots** are saved under **`<report_directory>/evalreport_plots/`** (same folder as your HTML/JSON/PDF file’s parent). So custom `output_path="my_reports/x.html"` → plots in `my_reports/evalreport_plots/`.
+
+### Task-specific API
+
+Useful when you want full control (e.g. set `output_dir` before `run_all()` so plots land next to a chosen folder):
+
+```python
+from pathlib import Path
+from evalreport import ClassificationReport, RegressionReport, RankingReport
+
+# Classification (binary or multiclass)
+cls = ClassificationReport(
+    y_true=[0, 1, 2, 0],
+    y_pred=[0, 2, 2, 0],
+    # y_prob: (n_samples, n_classes) for multiclass log loss / AUC
+    labels=[0, 1, 2],  # optional fixed class order for confusion matrix
+)
+cls.output_dir = Path("reports")  # optional; default for plots if set before run_all()
+cls.run_all()
+cls.save("reports/classification_report.html", format="html")
+cls.save("reports/classification_report.json", format="json")
+
+# Regression
+reg = RegressionReport(y_true=[1.0, 2.0, 3.0], y_pred=[1.1, 1.9, 3.2])
+reg.output_dir = Path("reports")
+reg.run_all()
+reg.save("reports/regression_report.pdf", format="pdf")  # needs reportlab
+
+# Recommendation / ranking
+rank = RankingReport(
+    relevant=[[1, 2], [3]],
+    ranked=[[1, 4, 5], [3, 1, 2]],
+    k_values=(1, 5, 10),
+)
+rank.output_dir = Path("reports")
+rank.run_all()
+rank.save("reports/ranking_report.html", format="html")
+```
+
+---
+
+## What each task includes
+
+### Classification
+
+| Area | Details |
+|------|--------|
+| **Metrics** | Accuracy; precision / recall / F1 (micro, macro, weighted); MCC; Cohen’s κ; log loss (with probs); ROC-AUC / PR-AUC when applicable; confusion matrix (table). |
+| **Plots** | Confusion matrix heatmap; **binary** ROC & PR curves when `y_prob` is provided. |
+| **Insights** | Class imbalance hint; most common misclassification pair. |
+| **HTML** | Styled layout: each metric with a short explanation, insights, and embedded plot images. |
+
+**Probabilities**
+
+- Binary: `y_prob` as length-`n` scores for the positive class, or shape `(n, 2)`.
+- Multiclass: `(n_samples, n_classes)` for log loss / multiclass AUC where supported.
+
+### Regression
+
+| Area | Details |
+|------|--------|
+| **Metrics** | MAE, MSE, RMSE, R², median absolute error, MAPE (where defined), mean error (bias). |
+| **Plots** | Residuals vs predicted, predicted vs actual, residual histogram. |
+| **Insights** | Over/under-prediction bias; heavy-tail error hint. |
+| **HTML** | Same rich layout as classification. |
+
+### Clustering
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `X` (feature matrix) and `labels` (cluster assignments) |
+| **Metrics** | Silhouette score, Davies–Bouldin index, Calinski–Harabasz score, cluster sizes |
+| **Plots** | Cluster scatter (PCA) and cluster size distribution |
+| **Insights** | Separability + imbalance hints |
+| **HTML** | Styled metrics/insights plus embedded plot images |
+
+### Time Series / Forecasting
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `y_true`, `y_pred`, and `timestamps` (same length) |
+| **Metrics** | MAE, MSE, RMSE, MAPE, SMAPE, mean forecast error, rolling RMSE summary |
+| **Plots** | Actual vs forecast, residuals over time, rolling RMSE over time |
+| **Insights** | Systematic bias and drift/stability hints via rolling RMSE |
+| **HTML** | Styled metrics/insights plus embedded plot images |
+
+### Recommendation / Ranking
+
+| Area | Details |
+|------|--------|
+| **Inputs** | `relevant`: ground-truth relevant **item IDs** per user (or query). `ranked`: **ordered** recommended lists per user (same length as `relevant`). |
+| **Metrics** | **MAP** (binary relevance), **Precision@K**, **Recall@K**, **NDCG@K**, **Hit Rate@K** for each K in `k_values` (default `(1, 5, 10)`). |
+| **Plots** | Precision@K curve; mean **cumulative gain** vs rank cutoff. |
+| **Insights** | Drop in precision at larger K; long-tail spread in \#relevant per user; low-MAP hint. |
+| **`generate_report`** | `task="recommendation"` / `"ranking"` / `"recommender"` with `y_true=relevant`, `y_pred=ranked`. |
+
+---
+
+## Output formats
+
+| Format | How | Notes |
+|--------|-----|--------|
+| **HTML** | `format="html"` or `.html` | Metrics + descriptions + insights + plot images. |
+| **JSON** | `format="json"` or `.json` | `metrics`, `insights`, `plots` (paths to PNGs). |
+| **Markdown** | `format="markdown"` or `.md` | Metrics and insights (no embedded images). |
+| **PDF** | `format="pdf"` or `.pdf` | Text summary (metrics + descriptions + insights); install `reportlab`. |
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/RAAHUL-tech/model-eval-toolkit.git
+cd model-eval-toolkit
+python -m venv .venv
+source .venv/bin/activate  # Windows: .venv\Scripts\activate
+pip install -e ".[test]"
+pytest -q
+# optional coverage
+pytest --cov=evalreport --cov-report=term-missing
+```
+
+Build and check the package:
+
+```bash
+pip install build twine
+python -m build
+twine check dist/*
+```
+
+### CI and PyPI releases
+
+GitHub Actions (`.github/workflows/ci.yml`):
+
+- **Pull requests** → runs **tests** only (Python 3.9–3.11).
+- **Push to `main`** (including when a PR is merged) → runs **tests**, then **publishes** to [PyPI](https://pypi.org/project/model-eval-toolkit/) if tests pass.
+
+**One-time setup**
+
+1. On [pypi.org](https://pypi.org/manage/account/token/), create an **API token** scoped to this project (or your whole account for a first publish).
+2. In the GitHub repo: **Settings → Secrets and variables → Actions → New repository secret**
+   - Name: `PYPI_API_TOKEN`
+   - Value: the token (often starts with `pypi-`).
+
+**Before each release**
+
+- Bump `version` in `pyproject.toml`. PyPI rejects re-uploading the same version.
+
+Optional: use [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (OIDC) and drop the token; the workflow already requests `id-token: write` for that path.
+
+---
+
+## Roadmap
+
+Additional task types (clustering, time series, ranking, NLP, CV) and a plugin-style API are planned. Issues and PRs welcome on [GitHub](https://github.com/RAAHUL-tech/model-eval-toolkit).
+
+---
+
+## License
+
+See [LICENSE](LICENSE).

model_eval_toolkit-0.1.0/evalreport/__init__.py

@@ -0,0 +1,28 @@
+from .core.entrypoints import generate_report
+from .classification.report import ClassificationReport
+from .regression.report import RegressionReport
+from .__version__ import __version__
+from .clustering.report import ClusteringReport
+from .timeseries.report import TimeSeriesReport
+from .nlp.text_classification import TextClassificationReport
+from .nlp.text_generation import TextGenerationReport
+from .vision.segmentation import SegmentationReport
+from .vision.detection import DetectionReport
+from .vision.image_classification import ImageClassificationReport
+from .ranking.report import RankingReport
+
+__all__ = [
+    "generate_report",
+    "__version__",
+    "ClassificationReport",
+    "RegressionReport",
+    "ClusteringReport",
+    "TimeSeriesReport",
+    "TextClassificationReport",
+    "TextGenerationReport",
+    "SegmentationReport",
+    "DetectionReport",
+    "ImageClassificationReport",
+    "RankingReport",
+]
+

model_eval_toolkit-0.1.0/evalreport/__version__.py

@@ -0,0 +1,2 @@
+__version__ = "0.1.0"
+

model_eval_toolkit-0.1.0/evalreport/classification/__init__.py

@@ -0,0 +1,4 @@
+from .report import ClassificationReport
+
+__all__ = ["ClassificationReport"]
+