evalsuite-python 0.1.0__tar.gz

evalsuite_python-0.1.0/.gitignore

@@ -0,0 +1,19 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+.env
+.coverage
+coverage.xml
+htmlcov/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.hypothesis/
+.DS_Store
+.idea/
+.vscode/

evalsuite_python-0.1.0/CHANGELOG.md

@@ -0,0 +1,82 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses
+[Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-08
+
+First stable release. Everything from the 0.1.0 roadmap: classification and regression metrics, result
+system, input validation, metric registry, model comparison with confidence intervals and paired tests,
+plots, HTML/CSV/LaTeX/Markdown reports, classification report, command-line tool and benchmarks.
+
+### Changed
+- Development status: Production/Stable.
+- README (PyPI description) now includes the benchmark table against scikit-learn.
+
+## [0.1.0b2]
+
+### Changed
+- `es.plot.calibration`: the legend now sits below the axes by default so it no longer covers the
+  curves; new `legend_loc` argument (`"below"` or any matplotlib location).
+
+## [0.1.0b1]
+
+Feature-complete for 0.1.0.
+
+### Added
+
+- Plots (`es.plot`, optional `[plot]` extra): ROC, precision-recall, calibration, confusion matrix, residuals /
+  predicted-vs-true, and model-comparison forest plots. Values come from EvalSuite's metrics; several models
+  get distinct colours and line styles. Importing `evalsuite` never imports matplotlib.
+- Reporting: `classification_report` (per-class precision, recall, F1, specificity, support; accuracy, micro,
+  macro and weighted averages; matches scikit-learn); `to_html()` and `to_csv()` on every result; `save(path)`
+  choosing the format from the extension (.json .csv .md .tex .html .txt).
+- `calibration_curve` and `expected_calibration_error` (matches scikit-learn's calibration curve).
+- Command line: `evalsuite evaluate | report | compare | plot | metrics | info | benchmark`, and
+  `python -m evalsuite`. Reads CSV, TSV, Parquet and JSON; clear one-line errors with exit code 2.
+- Benchmarks (`evalsuite.benchmarks.run_benchmarks`, `evalsuite benchmark`): time and peak memory against
+  scikit-learn, with a check that both libraries return the same numbers. Results in `BENCHMARKS.md`.
+
+### Changed
+
+- Regression `evaluate()` validates inputs once for all metrics and uses a faster unweighted mean (2× faster
+  on large arrays).
+
+## [0.1.0a2]
+
+### Added
+
+- Model comparison (`es.compare`): per-model confidence intervals from paired bootstrap resamples, pairwise
+  tests (McNemar for accuracy, DeLong for binary ROC AUC, paired bootstrap otherwise), multiple-comparison
+  correction, best model per metric, and summary/pandas/Markdown/LaTeX/JSON export.
+- Confidence intervals: `bootstrap_ci` (percentile, basic, BCa; stratified and reproducible),
+  `proportion_ci` and `accuracy_ci` (Wilson, Clopper-Pearson, normal), `roc_auc_ci` (DeLong).
+- Paired tests: `mcnemar_test`, `delong_test`, `paired_bootstrap_test`.
+- Effect sizes: `cohens_d` (independent and paired), `hedges_g`, `cliffs_delta`; `adjust_pvalues`
+  (Holm, Bonferroni, Benjamini-Hochberg, Benjamini-Yekutieli).
+- Reference tests against statsmodels and SciPy, brute-force DeLong checks and coverage simulations.
+- Python 3.14 support and CI.
+
+### Fixed
+
+- 0.1.0a1 installed an `evalsuite` console command although the CLI is not implemented yet, so the command
+  failed with `ModuleNotFoundError`. The entry point is removed until the CLI ships.
+
+## [0.1.0a1]
+
+First alpha, published to reserve the name and test the release pipeline. Distribution name
+`evalsuite-python` (`pip install --pre evalsuite-python`), imported as `evalsuite`.
+
+### Added
+
+- Core: input validation with actionable errors, exception hierarchy, immutable result objects with
+  JSON/pandas/Markdown/LaTeX export, metric registry (`list_metrics`, `metric_info`), and an evaluation
+  context that computes the confusion matrix once per evaluation.
+- Classification metrics for binary, multiclass and multilabel targets with all averaging modes and
+  sample weights.
+- Regression metrics for single- and multi-output targets with sample weights.
+- `evaluate()` high-level API with task inference and default metric sets.
+- Reference tests against scikit-learn and property-based tests; Python 3.9 to 3.13.

evalsuite_python-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,26 @@
+# Contributing to EvalSuite
+
+Thank you for helping. EvalSuite values correctness and clarity over breadth.
+
+## Adding or changing a metric
+
+1. Implement it with `@register(...)`: definition, formula, range, input requirements and at least one
+   reference.
+2. Validate inputs through `evalsuite.core.validation`; raise `InputValidationError` for bad inputs and
+   `MetricInputError` for values outside the metric's domain, with a message saying how to fix it.
+3. Test it: against an established implementation where definitions coincide (scikit-learn, SciPy), plus
+   hand-computed cases, edge cases and, where useful, property-based tests.
+4. Document conventions explicitly (averaging, label order, zero division) and update `CHANGELOG.md`.
+
+## Checks
+
+```bash
+pytest --cov=evalsuite      # coverage must stay at or above 95% for core modules
+ruff check . && ruff format --check .
+mypy                        # strict
+```
+
+## Reporting issues
+
+Include the EvalSuite, Python and NumPy versions, a minimal example, the expected result (with a reference)
+and the actual result.

evalsuite_python-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Manoj Kumar C S
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

evalsuite_python-0.1.0/PKG-INFO

@@ -0,0 +1,247 @@
+Metadata-Version: 2.5
+Name: evalsuite-python
+Version: 0.1.0
+Summary: A unified Python framework for machine-learning, clinical, statistical, segmentation, object-detection, uncertainty, and model evaluation.
+Project-URL: Homepage, https://evalsuite-nine.vercel.app
+Project-URL: Documentation, https://evalsuite-nine.vercel.app/docs
+Project-URL: Source, https://github.com/mkcs28/evalsuite-python
+Project-URL: Issues, https://github.com/mkcs28/evalsuite-python/issues
+Project-URL: Changelog, https://github.com/mkcs28/evalsuite-python/blob/main/CHANGELOG.md
+Author: Manoj Kumar C S
+Maintainer: Manoj Kumar C S
+License-Expression: MIT
+License-File: LICENSE
+Keywords: classification,evaluation,machine learning,metrics,regression,reproducibility,statistics
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.22
+Requires-Dist: pandas>=1.4
+Requires-Dist: scipy>=1.8
+Provides-Extra: all
+Requires-Dist: matplotlib>=3.5; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: hypothesis>=6.80; extra == 'dev'
+Requires-Dist: matplotlib>=3.5; extra == 'dev'
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pandas-stubs; extra == 'dev'
+Requires-Dist: pip-audit; extra == 'dev'
+Requires-Dist: pytest-cov>=4; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Requires-Dist: scikit-learn>=1.2; extra == 'dev'
+Requires-Dist: statsmodels>=0.13; extra == 'dev'
+Requires-Dist: twine; extra == 'dev'
+Provides-Extra: plot
+Requires-Dist: matplotlib>=3.5; extra == 'plot'
+Description-Content-Type: text/markdown
+
+# EvalSuite
+
+[![CI](https://github.com/mkcs28/evalsuite-python/actions/workflows/ci.yml/badge.svg)](https://github.com/mkcs28/evalsuite-python/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/evalsuite-python)](https://pypi.org/project/evalsuite-python/)
+[![Python](https://img.shields.io/pypi/pyversions/evalsuite-python)](https://pypi.org/project/evalsuite-python/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+**Unified, reproducible evaluation for machine learning and research.**
+
+EvalSuite brings classification and regression metrics (with clinical, statistical, segmentation and
+object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+
+> **Status: stable (0.1.0).** Every item on the 0.1.0 roadmap is implemented and verified.
+
+## Installation
+
+```bash
+pip install evalsuite-python
+```
+
+The package is installed as `evalsuite-python` and imported as `evalsuite`:
+
+```python
+import evalsuite as es
+```
+
+## Why EvalSuite
+
+- **One consistent API.** Every metric returns a result object that behaves like a number and exports to
+  JSON, pandas, Markdown and LaTeX.
+- **Explicit conventions.** Averaging, label order, the positive class and zero-division behaviour are stated
+  and recorded in every result, never silently assumed.
+- **Validated.** Each metric is tested against scikit-learn where definitions coincide, plus property-based
+  tests and edge cases.
+- **Documented.** Every metric carries its definition, formula, range, input requirements and references,
+  available programmatically through `metric_info()`.
+- **Efficient.** `evaluate()` validates inputs once and computes the confusion matrix once for all metrics.
+- **Lightweight.** Requires only NumPy, SciPy and pandas.
+
+## Quick start
+
+```python
+import evalsuite as es
+
+y_true = [0, 1, 1, 0, 1, 0]
+y_pred = [0, 1, 0, 0, 1, 1]
+y_prob = [0.1, 0.9, 0.4, 0.2, 0.8, 0.6]
+
+result = es.evaluate(y_true, y_pred, y_prob=y_prob)
+print(result.summary())
+
+result["f1"]  # MetricResult(f1=0.666667)
+f"{result['mcc']:.3f}"  # '0.333'
+result.to_latex(caption="Test-set performance")
+result.to_dataframe()
+
+es.f1(y_true, y_pred)  # individual metrics
+es.roc_auc(y_true, y_prob)
+es.metric_info("classification.mcc").formula  # documentation
+es.list_metrics("regression")
+```
+
+## Comparing models
+
+```python
+result = es.compare(
+    y_true,
+    {"logistic": pred_lr, "forest": pred_rf, "boosting": pred_gb},
+    probabilities={"logistic": prob_lr, "forest": prob_rf, "boosting": prob_gb},
+    random_state=0,
+)
+print(result.summary())  # estimates with 95% CIs, paired tests, Holm-adjusted p-values
+result.to_latex(label="tab:models")
+
+es.bootstrap_ci("f1", y_true, y_pred, average="macro", random_state=0)  # BCa interval for any metric
+es.accuracy_ci(y_true, y_pred)  # Wilson interval
+es.delong_test(y_true, prob_a, prob_b)  # two correlated AUCs
+es.mcnemar_test(y_true, pred_a, pred_b)
+```
+
+Every model is evaluated on the same bootstrap resamples, so differences are paired. Accuracy is compared
+with McNemar's test, binary ROC AUC with DeLong's test and other metrics with a paired bootstrap test;
+p-values are adjusted for multiple comparisons (Holm by default).
+
+## Classification report
+
+```python
+report = es.classification_report(y_true, y_pred)
+print(report)  # per-class precision, recall, F1, specificity, support + averages
+report.save("report.html")  # also .csv .md .tex .json .txt
+```
+
+## Plots
+
+```bash
+pip install "evalsuite-python[plot]"   # adds matplotlib; importing evalsuite never loads it
+```
+
+```python
+es.plot.roc(y_true, {"logistic": prob_lr, "forest": prob_rf})  # AUC in the legend
+es.plot.pr(y_true, prob)  # AP and the prevalence line
+es.plot.calibration(y_true, prob)  # reliability diagram, ECE, Brier
+es.plot.confusion_matrix(y_true, y_pred, normalize="true")
+es.plot.residuals(y_reg, pred_reg)  # or kind="predicted"
+es.plot.comparison(es.compare(...))  # forest plot with CIs
+```
+
+Each function returns a matplotlib `Axes` (pass `ax=` to draw into your own figure). The numbers shown are
+computed with EvalSuite's metrics, so plots and tables always agree. Several models get distinct colours
+*and* line styles, so figures stay readable in greyscale print.
+
+## Exports
+
+Every result (`evaluate`, `classification_report`, `compare`, single metrics) exports to `summary()`,
+`to_json()`, `to_csv()`, `to_dataframe()`, `to_markdown()`, `to_latex()` and `to_html()`, and `save(path)` picks
+the format from the extension. HTML pages are standalone (inline CSS, no scripts) and escape all text.
+
+## Command line
+
+```bash
+evalsuite evaluate predictions.csv --y-true label --y-pred pred --y-prob prob
+evalsuite report predictions.csv --y-true label --y-pred pred -o report.html
+evalsuite compare predictions.csv --y-true label --pred lr=pred_lr --pred rf=pred_rf \
+    --prob lr=p_lr --prob rf=p_rf --plot comparison.png
+evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
+evalsuite metrics --category classification
+evalsuite info classification.mcc
+evalsuite benchmark --quick
+```
+
+Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` or the `-o` extension
+(text, json, csv, markdown, latex, html). Errors are reported in one line with exit code 2.
+
+## Performance
+
+Benchmarked against scikit-learn on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
+scikit-learn 1.9, Linux x86_64). Every result agrees with scikit-learn to floating-point rounding
+(largest difference 1.1e-16).
+
+| Case | n | EvalSuite (ms) | scikit-learn (ms) | Speed-up | Peak memory EvalSuite / sklearn (MiB) |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| 8 binary label metrics via `evaluate()` | 1,000 | 0.38 | 11.70 | **31.2×** | 0.04 / 0.05 |
+| 8 binary label metrics via `evaluate()` | 100,000 | 10.9 | 116.9 | **10.7×** | 3.2 / 3.1 |
+| 8 binary label metrics via `evaluate()` | 1,000,000 | 108.8 | 1043.6 | **9.6×** | 31.5 / 30.5 |
+| macro F1, 10 classes | 1,000,000 | 88.4 | 139.3 | **1.58×** | 30.5 / 21.8 |
+| ROC AUC, binary | 1,000,000 | 247.4 | 352.5 | **1.42×** | 91.6 / 76.3 |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000 | 0.12 | 0.90 | **7.2×** | 0.03 / 0.02 |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | 29.3 | 20.1 | 0.69× | 22.9 / 15.3 |
+
+`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where the
+speed-up comes from. Large regression arrays are slower because EvalSuite checks every value for NaN,
+infinity, shape and dtype before computing. Reproduce on your machine with `evalsuite benchmark`; full
+table and notes in
+[BENCHMARKS.md](https://github.com/mkcs28/evalsuite-python/blob/main/BENCHMARKS.md).
+
+## Metrics in this release
+
+**Classification** (binary, multiclass, multilabel; micro/macro/weighted/samples/per-class averaging;
+sample weights): accuracy, balanced accuracy, precision, recall, specificity, NPV, F1, F-beta, Jaccard,
+MCC, Cohen's kappa (unweighted, linear, quadratic), Hamming loss, confusion matrix, ROC AUC (binary,
+one-vs-rest, one-vs-one), average precision, ROC and PR curves, log loss, Brier score, top-k accuracy,
+calibration curve and expected calibration error.
+
+**Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
+MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
+loss, Huber loss, relative absolute error, relative squared error.
+
+## Conventions
+
+- `average="auto"` resolves to `"binary"` for binary targets and `"macro"` otherwise; the resolved value
+  is stored in `result.params["average"]`.
+- Labels are sorted unless you pass `labels=[...]`; that order defines per-class outputs and the columns
+  of 2-D `y_prob`.
+- Undefined ratios (zero denominators) return 0 **with an `UndefinedMetricWarning`**; pass
+  `zero_division=np.nan` to propagate NaN, or `0`/`1` to choose silently.
+- Domain violations raise clear errors instead of being patched over (for example MAPE with zero targets).
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest --cov=evalsuite
+ruff check . && ruff format --check . && mypy
+```
+
+## Links
+
+- PyPI: https://pypi.org/project/evalsuite-python/
+- Website and documentation: https://evalsuite-nine.vercel.app
+- Website source: https://github.com/mkcs28/evalsuite
+
+## License
+
+MIT. See [LICENSE](LICENSE).

evalsuite_python-0.1.0/README.md

@@ -0,0 +1,195 @@
+# EvalSuite
+
+[![CI](https://github.com/mkcs28/evalsuite-python/actions/workflows/ci.yml/badge.svg)](https://github.com/mkcs28/evalsuite-python/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/evalsuite-python)](https://pypi.org/project/evalsuite-python/)
+[![Python](https://img.shields.io/pypi/pyversions/evalsuite-python)](https://pypi.org/project/evalsuite-python/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+**Unified, reproducible evaluation for machine learning and research.**
+
+EvalSuite brings classification and regression metrics (with clinical, statistical, segmentation and
+object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+
+> **Status: stable (0.1.0).** Every item on the 0.1.0 roadmap is implemented and verified.
+
+## Installation
+
+```bash
+pip install evalsuite-python
+```
+
+The package is installed as `evalsuite-python` and imported as `evalsuite`:
+
+```python
+import evalsuite as es
+```
+
+## Why EvalSuite
+
+- **One consistent API.** Every metric returns a result object that behaves like a number and exports to
+  JSON, pandas, Markdown and LaTeX.
+- **Explicit conventions.** Averaging, label order, the positive class and zero-division behaviour are stated
+  and recorded in every result, never silently assumed.
+- **Validated.** Each metric is tested against scikit-learn where definitions coincide, plus property-based
+  tests and edge cases.
+- **Documented.** Every metric carries its definition, formula, range, input requirements and references,
+  available programmatically through `metric_info()`.
+- **Efficient.** `evaluate()` validates inputs once and computes the confusion matrix once for all metrics.
+- **Lightweight.** Requires only NumPy, SciPy and pandas.
+
+## Quick start
+
+```python
+import evalsuite as es
+
+y_true = [0, 1, 1, 0, 1, 0]
+y_pred = [0, 1, 0, 0, 1, 1]
+y_prob = [0.1, 0.9, 0.4, 0.2, 0.8, 0.6]
+
+result = es.evaluate(y_true, y_pred, y_prob=y_prob)
+print(result.summary())
+
+result["f1"]  # MetricResult(f1=0.666667)
+f"{result['mcc']:.3f}"  # '0.333'
+result.to_latex(caption="Test-set performance")
+result.to_dataframe()
+
+es.f1(y_true, y_pred)  # individual metrics
+es.roc_auc(y_true, y_prob)
+es.metric_info("classification.mcc").formula  # documentation
+es.list_metrics("regression")
+```
+
+## Comparing models
+
+```python
+result = es.compare(
+    y_true,
+    {"logistic": pred_lr, "forest": pred_rf, "boosting": pred_gb},
+    probabilities={"logistic": prob_lr, "forest": prob_rf, "boosting": prob_gb},
+    random_state=0,
+)
+print(result.summary())  # estimates with 95% CIs, paired tests, Holm-adjusted p-values
+result.to_latex(label="tab:models")
+
+es.bootstrap_ci("f1", y_true, y_pred, average="macro", random_state=0)  # BCa interval for any metric
+es.accuracy_ci(y_true, y_pred)  # Wilson interval
+es.delong_test(y_true, prob_a, prob_b)  # two correlated AUCs
+es.mcnemar_test(y_true, pred_a, pred_b)
+```
+
+Every model is evaluated on the same bootstrap resamples, so differences are paired. Accuracy is compared
+with McNemar's test, binary ROC AUC with DeLong's test and other metrics with a paired bootstrap test;
+p-values are adjusted for multiple comparisons (Holm by default).
+
+## Classification report
+
+```python
+report = es.classification_report(y_true, y_pred)
+print(report)  # per-class precision, recall, F1, specificity, support + averages
+report.save("report.html")  # also .csv .md .tex .json .txt
+```
+
+## Plots
+
+```bash
+pip install "evalsuite-python[plot]"   # adds matplotlib; importing evalsuite never loads it
+```
+
+```python
+es.plot.roc(y_true, {"logistic": prob_lr, "forest": prob_rf})  # AUC in the legend
+es.plot.pr(y_true, prob)  # AP and the prevalence line
+es.plot.calibration(y_true, prob)  # reliability diagram, ECE, Brier
+es.plot.confusion_matrix(y_true, y_pred, normalize="true")
+es.plot.residuals(y_reg, pred_reg)  # or kind="predicted"
+es.plot.comparison(es.compare(...))  # forest plot with CIs
+```
+
+Each function returns a matplotlib `Axes` (pass `ax=` to draw into your own figure). The numbers shown are
+computed with EvalSuite's metrics, so plots and tables always agree. Several models get distinct colours
+*and* line styles, so figures stay readable in greyscale print.
+
+## Exports
+
+Every result (`evaluate`, `classification_report`, `compare`, single metrics) exports to `summary()`,
+`to_json()`, `to_csv()`, `to_dataframe()`, `to_markdown()`, `to_latex()` and `to_html()`, and `save(path)` picks
+the format from the extension. HTML pages are standalone (inline CSS, no scripts) and escape all text.
+
+## Command line
+
+```bash
+evalsuite evaluate predictions.csv --y-true label --y-pred pred --y-prob prob
+evalsuite report predictions.csv --y-true label --y-pred pred -o report.html
+evalsuite compare predictions.csv --y-true label --pred lr=pred_lr --pred rf=pred_rf \
+    --prob lr=p_lr --prob rf=p_rf --plot comparison.png
+evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
+evalsuite metrics --category classification
+evalsuite info classification.mcc
+evalsuite benchmark --quick
+```
+
+Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` or the `-o` extension
+(text, json, csv, markdown, latex, html). Errors are reported in one line with exit code 2.
+
+## Performance
+
+Benchmarked against scikit-learn on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
+scikit-learn 1.9, Linux x86_64). Every result agrees with scikit-learn to floating-point rounding
+(largest difference 1.1e-16).
+
+| Case | n | EvalSuite (ms) | scikit-learn (ms) | Speed-up | Peak memory EvalSuite / sklearn (MiB) |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| 8 binary label metrics via `evaluate()` | 1,000 | 0.38 | 11.70 | **31.2×** | 0.04 / 0.05 |
+| 8 binary label metrics via `evaluate()` | 100,000 | 10.9 | 116.9 | **10.7×** | 3.2 / 3.1 |
+| 8 binary label metrics via `evaluate()` | 1,000,000 | 108.8 | 1043.6 | **9.6×** | 31.5 / 30.5 |
+| macro F1, 10 classes | 1,000,000 | 88.4 | 139.3 | **1.58×** | 30.5 / 21.8 |
+| ROC AUC, binary | 1,000,000 | 247.4 | 352.5 | **1.42×** | 91.6 / 76.3 |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000 | 0.12 | 0.90 | **7.2×** | 0.03 / 0.02 |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | 29.3 | 20.1 | 0.69× | 22.9 / 15.3 |
+
+`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where the
+speed-up comes from. Large regression arrays are slower because EvalSuite checks every value for NaN,
+infinity, shape and dtype before computing. Reproduce on your machine with `evalsuite benchmark`; full
+table and notes in
+[BENCHMARKS.md](https://github.com/mkcs28/evalsuite-python/blob/main/BENCHMARKS.md).
+
+## Metrics in this release
+
+**Classification** (binary, multiclass, multilabel; micro/macro/weighted/samples/per-class averaging;
+sample weights): accuracy, balanced accuracy, precision, recall, specificity, NPV, F1, F-beta, Jaccard,
+MCC, Cohen's kappa (unweighted, linear, quadratic), Hamming loss, confusion matrix, ROC AUC (binary,
+one-vs-rest, one-vs-one), average precision, ROC and PR curves, log loss, Brier score, top-k accuracy,
+calibration curve and expected calibration error.
+
+**Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
+MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
+loss, Huber loss, relative absolute error, relative squared error.
+
+## Conventions
+
+- `average="auto"` resolves to `"binary"` for binary targets and `"macro"` otherwise; the resolved value
+  is stored in `result.params["average"]`.
+- Labels are sorted unless you pass `labels=[...]`; that order defines per-class outputs and the columns
+  of 2-D `y_prob`.
+- Undefined ratios (zero denominators) return 0 **with an `UndefinedMetricWarning`**; pass
+  `zero_division=np.nan` to propagate NaN, or `0`/`1` to choose silently.
+- Domain violations raise clear errors instead of being patched over (for example MAPE with zero targets).
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest --cov=evalsuite
+ruff check . && ruff format --check . && mypy
+```
+
+## Links
+
+- PyPI: https://pypi.org/project/evalsuite-python/
+- Website and documentation: https://evalsuite-nine.vercel.app
+- Website source: https://github.com/mkcs28/evalsuite
+
+## License
+
+MIT. See [LICENSE](LICENSE).