evalsuite-python 0.2.1__tar.gz → 0.3.1__tar.gz

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/CHANGELOG.md

@@ -6,6 +6,65 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.1] - 2026-10-09
+
+Quality-assurance release: release pipeline, packaging checks, edge-case fixes and an overall benchmark.
+No API was removed or renamed; code written for 0.3.0 runs unchanged.
+
+### Fixed
+- Labels that mix numbers and strings (`[0, "a"]`, or string `y_true` with integer `y_pred`) now raise
+  `InputValidationError` instead of being silently compared as text.
+- `rmse` no longer overflows to `inf` when errors exceed ~1e154; the value is computed with rescaling.
+- `bootstrap_ci`, `paired_bootstrap_test` and `compare` raise `StatisticalTestError` for fewer than two
+  observations instead of returning a zero-width interval.
+- A 2-D mask given as a nested Python list (`[[1, 1], [0, 0]]`) is read as one image, as a NumPy array is.
+- Release workflow: the GitHub release step no longer fails when the release already exists (for example
+  when the tag was created from the GitHub UI); it attaches the files instead.
+
+### Added
+- Benchmarks: an overall summary first (per suite and in total: cases, agreement with the reference,
+  how many are faster, geometric-mean and range of speed-ups) and rows in alphabetical order;
+  `BenchmarkResult.overall()`, `overall_markdown()` and `sorted_rows()`; `overall` in the JSON export.
+- Release workflow: refuses a tag that does not match the package version or a version already on PyPI,
+  checks the built files, uses `--verify-tag`, and can be run by hand to attach a version's PyPI files
+  (hash-verified) to its GitHub release.
+- CI: wheel and sdist installed in clean environments with `pip check` and a smoke test that must give the
+  same results for both, a distribution check (metadata, forbidden files, credentials), a minimum-dependency
+  job (NumPy 1.22, SciPy 1.8, pandas 1.4), README examples run as tests, and repository hygiene tests
+  (no credentials, no unsafe calls, least-privilege workflows, single version source).
+- QA tests against scikit-learn and statsmodels for the edge cases in the release checklist.
+- `SECURITY.md` with the reporting process.
+
+## [0.3.0] - 2026-10-09
+
+Computer vision (the v0.3.0 roadmap), with comparison, plots, reporting and benchmarks extended to it.
+
+### Added
+- Segmentation: `dice`, `iou`, `miou`, `pixel_accuracy`, `mean_pixel_accuracy`, `boundary_iou`,
+  `hausdorff_distance` (HD and HD95, anisotropic `spacing`), `average_surface_distance`,
+  `segmentation_confusion`, `per_image_scores` and `segmentation_report`. 2-D and 3-D masks, lists of
+  differently sized masks, `ignore_index`, `aggregate="dataset"|"image"`, undefined classes reported as NaN.
+- Object detection: `box_iou`, `detection_report` (the 12 COCO numbers plus AP per class),
+  `mean_average_precision`, `average_precision_detection` (COCO or VOC interpolation), `detection_pr_curve`,
+  `from_coco`. Matches pycocotools exactly, including crowd regions, area ranges and detection limits.
+- Comparison for vision: `compare`, `bootstrap_ci` and `paired_bootstrap_test` resample images for
+  segmentation masks and per-image detections.
+- Plots: `es.plot.segmentation` (prediction fill vs truth outline), `es.plot.per_class` (any per-class
+  result, a segmentation report or a detection report), `es.plot.detection_pr`.
+- CLI: `evalsuite segmentation` (.npy/.npz or a folder of mask images) and `evalsuite detection` (COCO JSON).
+- Benchmarks: `--suite vision` against scikit-learn, SciPy and pycocotools.
+- `vision` extra (Pillow, for reading mask image folders); `CITATION.cff`.
+
+### Changed
+- `evaluate()` points segmentation masks and detection annotations to the right functions instead of
+  failing with a shape error.
+- CI and release workflows use the Node 24 versions of the GitHub actions.
+
+### Fixed
+- Calibration slope and intercept no longer emit an overflow warning before reporting a perfectly
+  separated outcome.
+- The CLI no longer fails on consoles that cannot print Unicode (Windows code pages).
+
 ## [0.2.1] - 2026-10-09
 
 ### Added

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: evalsuite-python
-Version: 0.2.1
+Version: 0.3.1
 Summary: A unified Python framework for machine-learning, clinical, statistical, segmentation, object-detection, uncertainty, and model evaluation.
 Project-URL: Homepage, https://evalsuite-nine.vercel.app
 Project-URL: Documentation, https://evalsuite-nine.vercel.app/docs
@@ -11,7 +11,7 @@ Author: Manoj Kumar C S, Nikhil D Bharadwaj
 Maintainer: Manoj Kumar C S, Nikhil D Bharadwaj
 License-Expression: MIT
 License-File: LICENSE
-Keywords: calibration,classification,clinical,diagnostic accuracy,evaluation,machine learning,metrics,regression,reproducibility,statistical tests,statistics
+Keywords: calibration,classification,clinical,diagnostic accuracy,evaluation,mAP,machine learning,metrics,object detection,regression,reproducibility,segmentation,statistical tests,statistics
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Healthcare Industry
@@ -27,6 +27,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Scientific/Engineering
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
 Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
 Classifier: Typing :: Typed
 Requires-Python: >=3.9
@@ -35,21 +36,26 @@ Requires-Dist: pandas>=1.4
 Requires-Dist: scipy>=1.8
 Provides-Extra: all
 Requires-Dist: matplotlib>=3.5; extra == 'all'
+Requires-Dist: pillow>=9; extra == 'all'
 Provides-Extra: dev
 Requires-Dist: build; extra == 'dev'
 Requires-Dist: hypothesis>=6.80; extra == 'dev'
 Requires-Dist: matplotlib>=3.5; extra == 'dev'
 Requires-Dist: mypy>=1.10; extra == 'dev'
 Requires-Dist: pandas-stubs; extra == 'dev'
+Requires-Dist: pillow>=9; extra == 'dev'
 Requires-Dist: pip-audit; extra == 'dev'
+Requires-Dist: pycocotools>=2.0.7; extra == 'dev'
 Requires-Dist: pytest-cov>=4; extra == 'dev'
 Requires-Dist: pytest>=7; extra == 'dev'
 Requires-Dist: ruff>=0.6; extra == 'dev'
-Requires-Dist: scikit-learn>=1.2; extra == 'dev'
+Requires-Dist: scikit-learn>=1.4; extra == 'dev'
 Requires-Dist: statsmodels>=0.13; extra == 'dev'
 Requires-Dist: twine; extra == 'dev'
 Provides-Extra: plot
 Requires-Dist: matplotlib>=3.5; extra == 'plot'
+Provides-Extra: vision
+Requires-Dist: pillow>=9; extra == 'vision'
 Description-Content-Type: text/markdown
 
 # EvalSuite
@@ -61,10 +67,10 @@ Description-Content-Type: text/markdown
 
 **Unified, reproducible evaluation for machine learning and research.**
 
-EvalSuite brings classification, regression, clinical and statistical evaluation (with segmentation and
-object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+EvalSuite brings classification, regression, clinical, statistical, segmentation and object-detection
+evaluation into one consistent, validated, documented framework.
 
-> **Status: stable (0.2.1).** Every item on the 0.1.0 and 0.2.0 roadmaps is implemented and verified.
+> **Status: stable (0.3.1).** Every item on the 0.1.0, 0.2.0 and 0.3.0 roadmaps is implemented and verified.
 
 ## Installation
 
@@ -181,6 +187,45 @@ es.adjust_pvalues(p_values, method="hochberg")  # also holm, bonferroni, bh, by
 Every test returns a `TestResult` with the statistic, p-value and an effect size, computed with SciPy and
 checked against SciPy and statsmodels in the test suite.
 
+## Segmentation
+
+```python
+# label masks: one 2-D image, or images on the first axis (N, H, W) / (N, D, H, W), or a list of masks
+es.dice(y_true, y_pred)  # macro over classes, pixel counts summed over the dataset
+es.iou(y_true, y_pred, average=None)  # per class; classes absent from both masks are NaN, not 0
+es.miou(y_true, y_pred, ignore_index=255)
+es.dice(y_true, y_pred, aggregate="image")  # mean of per-image scores (medical imaging convention)
+es.boundary_iou(y_true, y_pred)  # Cheng et al. 2021
+es.hausdorff_distance(y_true, y_pred, percentile=95, spacing=(0.8, 0.8))  # HD95 in mm
+
+report = es.segmentation_report(y_true, y_pred, class_names={0: "background", 1: "liver"})
+print(report)  # mIoU, Dice, pixel accuracy, Boundary IoU, HD95, ASSD + per-class table
+es.plot.segmentation(image, y_true[0], y_pred[0])  # prediction fill, truth outline
+es.plot.per_class(report, metric="iou")  # per-class bars (also takes a detection report)
+```
+
+## Object detection
+
+```python
+y_true = [{"boxes": [[x1, y1, x2, y2], ...], "labels": [3, ...]}, ...]  # one dict per image
+y_pred = [{"boxes": [...], "labels": [...], "scores": [...]}, ...]
+
+report = es.detection_report(y_true, y_pred)  # the 12 COCO numbers + AP per class
+report["map"], report["map_50"], report["mar_100"]
+es.mean_average_precision(y_true, y_pred, iou_threshold=0.5)  # mAP@.50
+es.average_precision_detection(y_true, y_pred, interpolation="voc")  # per class, VOC-style
+es.box_iou(boxes_a, boxes_b, box_format="xywh")
+y_true, y_pred = es.from_coco("instances_val.json", "detections.json")
+es.plot.detection_pr(y_true, y_pred)
+```
+
+The COCO protocol (crowd regions, area ranges, max detections, 101-point interpolation) matches
+`pycocotools` to the last digit in the test suite.
+
+Models can be compared over the same images with intervals and paired tests, as for every other task:
+`es.compare(y_true, {"unet": masks_a, "deeplab": masks_b})` resamples images; for detection it compares
+mAP.
+
 ## Classification report
 
 ```python
@@ -225,6 +270,8 @@ evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
 evalsuite diagnostic predictions.csv --y-true label --y-pred pred   # sensitivity, LR+, DOR... with CIs
 evalsuite calibration predictions.csv --y-true label --y-prob prob  # slope, intercept, ECE, HL
 evalsuite plot decision predictions.csv --y-true label --y-prob prob -o dca.png
+evalsuite segmentation true_masks.npy pred_masks.npy --ignore-index 255 --plot per_class.png
+evalsuite detection instances_val.json detections.json --plot pr_curves.png
 evalsuite metrics --category clinical
 evalsuite info classification.mcc
 evalsuite benchmark --quick
@@ -236,21 +283,28 @@ Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` o
 ## Performance
 
 Benchmarked against reference implementations on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
-Linux x86_64). Every result agrees with the reference to floating-point rounding (largest difference
-1.4e-14).
-
-| Case | n | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
-| --- | ---: | --- | ---: | ---: | ---: |
-| 8 binary label metrics via `evaluate()` | 1,000,000 | scikit-learn | 30.2 | 1020.2 | **33.8×** |
-| macro F1, 10 classes | 1,000,000 | scikit-learn | 22.4 | 128.7 | **5.8×** |
-| ROC AUC, binary | 1,000,000 | scikit-learn | 173.2 | 300.6 | **1.7×** |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | scikit-learn | 19.1 | 9.7 | 0.51× |
-| sensitivity, specificity, LR+, LR− | 1,000,000 | scikit-learn | 66.2 | 392.8 | **5.9×** |
-| calibration slope and intercept | 1,000,000 | statsmodels | 178.0 | 1014.8 | **5.7×** |
-| decision curve, 99 thresholds | 1,000,000 | NumPy loop | 155.2 | 174.3 | **1.1×** |
-| diagnostic report (7 CIs) | 1,000,000 | statsmodels | 16.6 | 4.6 | 0.28× |
-| Welch t-test | 1,000,000 | SciPy | 14.5 | 7.3 | 0.51× |
-| Hochberg correction | 1,000,000 | statsmodels | 75.4 | 81.9 | **1.1×** |
+Linux x86_64). **Overall: 45 of 45 rows agree with the reference** (largest
+difference 2.8e-14), 32 are faster, and the geometric-mean speed-up across all
+15 cases and three sizes is **2.22×**. At 1,000,000 samples (pixels for
+segmentation; 1,000 images for detection and 50 for Hausdorff), in alphabetical order:
+
+| Case | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
+| --- | --- | ---: | ---: | ---: |
+| 10 classes: macro F1 | scikit-learn | 29.3 | 130.8 | **4.5×** |
+| binary: 8 label metrics via evaluate() | scikit-learn | 39.5 | 1084.1 | **27.4×** |
+| binary: ROC AUC | scikit-learn | 195.0 | 348.3 | **1.8×** |
+| calibration: slope and intercept | statsmodels | 193.0 | 1068.0 | **5.5×** |
+| clinical: diagnostic report (7 CIs) | statsmodels | 23.5 | 4.7 | 0.20× |
+| clinical: sensitivity, specificity, LR+, LR− | scikit-learn | 69.1 | 414.0 | **6.0×** |
+| decision curve: 99 thresholds | NumPy loop | 156.3 | 181.6 | **1.2×** |
+| detection: COCO evaluation (1000 images) | pycocotools | 817.3 | 983.7 | **1.2×** |
+| multiple testing: Hochberg (n p-values) | statsmodels | 82.0 | 108.0 | **1.3×** |
+| regression: MAE, MSE, RMSE, R² via evaluate() | scikit-learn | 23.1 | 10.8 | 0.47× |
+| segmentation: Dice and IoU per class (n = pixels) | scikit-learn | 43.0 | 290.0 | **6.8×** |
+| segmentation: Hausdorff distance (50 images) | SciPy | 27.0 | 20.2 | 0.75× |
+| statistics: Cramér's V (5×5 table) | SciPy | 0.5 | 0.4 | 0.88× |
+| statistics: Mann–Whitney U | SciPy | 349.7 | 350.2 | **1.0×** |
+| statistics: Welch t-test | SciPy | 14.5 | 7.2 | 0.49× |
 
 `evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where most
 of the speed-up comes from. Hypothesis tests use SciPy underneath, so they match its speed at best; rows
@@ -279,6 +333,13 @@ Kruskal–Wallis, Friedman, Shapiro–Wilk, χ², Fisher's exact; effect sizes (
 delta, Cramér's V); multiple-testing corrections (Bonferroni, Holm, Hochberg, Benjamini–Hochberg,
 Benjamini–Yekutieli).
 
+**Segmentation** (2-D and 3-D label masks; `ignore_index`; dataset or per-image aggregation): Dice, IoU,
+mIoU, pixel accuracy, mean pixel accuracy, Boundary IoU, Hausdorff distance and HD95, average symmetric
+surface distance (with pixel spacing), confusion matrix and a full report.
+
+**Object detection**: box IoU (xyxy, xywh, cxcywh), COCO mAP@[.50:.95], mAP@.50, mAP@.75, mAP and mAR by
+object size, AP per class with COCO or VOC interpolation, precision-recall curves, COCO file import.
+
 **Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
 MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
 loss, Huber loss, relative absolute error, relative squared error.

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/README.md

@@ -7,10 +7,10 @@
 
 **Unified, reproducible evaluation for machine learning and research.**
 
-EvalSuite brings classification, regression, clinical and statistical evaluation (with segmentation and
-object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+EvalSuite brings classification, regression, clinical, statistical, segmentation and object-detection
+evaluation into one consistent, validated, documented framework.
 
-> **Status: stable (0.2.1).** Every item on the 0.1.0 and 0.2.0 roadmaps is implemented and verified.
+> **Status: stable (0.3.1).** Every item on the 0.1.0, 0.2.0 and 0.3.0 roadmaps is implemented and verified.
 
 ## Installation
 
@@ -127,6 +127,45 @@ es.adjust_pvalues(p_values, method="hochberg")  # also holm, bonferroni, bh, by
 Every test returns a `TestResult` with the statistic, p-value and an effect size, computed with SciPy and
 checked against SciPy and statsmodels in the test suite.
 
+## Segmentation
+
+```python
+# label masks: one 2-D image, or images on the first axis (N, H, W) / (N, D, H, W), or a list of masks
+es.dice(y_true, y_pred)  # macro over classes, pixel counts summed over the dataset
+es.iou(y_true, y_pred, average=None)  # per class; classes absent from both masks are NaN, not 0
+es.miou(y_true, y_pred, ignore_index=255)
+es.dice(y_true, y_pred, aggregate="image")  # mean of per-image scores (medical imaging convention)
+es.boundary_iou(y_true, y_pred)  # Cheng et al. 2021
+es.hausdorff_distance(y_true, y_pred, percentile=95, spacing=(0.8, 0.8))  # HD95 in mm
+
+report = es.segmentation_report(y_true, y_pred, class_names={0: "background", 1: "liver"})
+print(report)  # mIoU, Dice, pixel accuracy, Boundary IoU, HD95, ASSD + per-class table
+es.plot.segmentation(image, y_true[0], y_pred[0])  # prediction fill, truth outline
+es.plot.per_class(report, metric="iou")  # per-class bars (also takes a detection report)
+```
+
+## Object detection
+
+```python
+y_true = [{"boxes": [[x1, y1, x2, y2], ...], "labels": [3, ...]}, ...]  # one dict per image
+y_pred = [{"boxes": [...], "labels": [...], "scores": [...]}, ...]
+
+report = es.detection_report(y_true, y_pred)  # the 12 COCO numbers + AP per class
+report["map"], report["map_50"], report["mar_100"]
+es.mean_average_precision(y_true, y_pred, iou_threshold=0.5)  # mAP@.50
+es.average_precision_detection(y_true, y_pred, interpolation="voc")  # per class, VOC-style
+es.box_iou(boxes_a, boxes_b, box_format="xywh")
+y_true, y_pred = es.from_coco("instances_val.json", "detections.json")
+es.plot.detection_pr(y_true, y_pred)
+```
+
+The COCO protocol (crowd regions, area ranges, max detections, 101-point interpolation) matches
+`pycocotools` to the last digit in the test suite.
+
+Models can be compared over the same images with intervals and paired tests, as for every other task:
+`es.compare(y_true, {"unet": masks_a, "deeplab": masks_b})` resamples images; for detection it compares
+mAP.
+
 ## Classification report
 
 ```python
@@ -171,6 +210,8 @@ evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
 evalsuite diagnostic predictions.csv --y-true label --y-pred pred   # sensitivity, LR+, DOR... with CIs
 evalsuite calibration predictions.csv --y-true label --y-prob prob  # slope, intercept, ECE, HL
 evalsuite plot decision predictions.csv --y-true label --y-prob prob -o dca.png
+evalsuite segmentation true_masks.npy pred_masks.npy --ignore-index 255 --plot per_class.png
+evalsuite detection instances_val.json detections.json --plot pr_curves.png
 evalsuite metrics --category clinical
 evalsuite info classification.mcc
 evalsuite benchmark --quick
@@ -182,21 +223,28 @@ Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` o
 ## Performance
 
 Benchmarked against reference implementations on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
-Linux x86_64). Every result agrees with the reference to floating-point rounding (largest difference
-1.4e-14).
-
-| Case | n | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
-| --- | ---: | --- | ---: | ---: | ---: |
-| 8 binary label metrics via `evaluate()` | 1,000,000 | scikit-learn | 30.2 | 1020.2 | **33.8×** |
-| macro F1, 10 classes | 1,000,000 | scikit-learn | 22.4 | 128.7 | **5.8×** |
-| ROC AUC, binary | 1,000,000 | scikit-learn | 173.2 | 300.6 | **1.7×** |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | scikit-learn | 19.1 | 9.7 | 0.51× |
-| sensitivity, specificity, LR+, LR− | 1,000,000 | scikit-learn | 66.2 | 392.8 | **5.9×** |
-| calibration slope and intercept | 1,000,000 | statsmodels | 178.0 | 1014.8 | **5.7×** |
-| decision curve, 99 thresholds | 1,000,000 | NumPy loop | 155.2 | 174.3 | **1.1×** |
-| diagnostic report (7 CIs) | 1,000,000 | statsmodels | 16.6 | 4.6 | 0.28× |
-| Welch t-test | 1,000,000 | SciPy | 14.5 | 7.3 | 0.51× |
-| Hochberg correction | 1,000,000 | statsmodels | 75.4 | 81.9 | **1.1×** |
+Linux x86_64). **Overall: 45 of 45 rows agree with the reference** (largest
+difference 2.8e-14), 32 are faster, and the geometric-mean speed-up across all
+15 cases and three sizes is **2.22×**. At 1,000,000 samples (pixels for
+segmentation; 1,000 images for detection and 50 for Hausdorff), in alphabetical order:
+
+| Case | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
+| --- | --- | ---: | ---: | ---: |
+| 10 classes: macro F1 | scikit-learn | 29.3 | 130.8 | **4.5×** |
+| binary: 8 label metrics via evaluate() | scikit-learn | 39.5 | 1084.1 | **27.4×** |
+| binary: ROC AUC | scikit-learn | 195.0 | 348.3 | **1.8×** |
+| calibration: slope and intercept | statsmodels | 193.0 | 1068.0 | **5.5×** |
+| clinical: diagnostic report (7 CIs) | statsmodels | 23.5 | 4.7 | 0.20× |
+| clinical: sensitivity, specificity, LR+, LR− | scikit-learn | 69.1 | 414.0 | **6.0×** |
+| decision curve: 99 thresholds | NumPy loop | 156.3 | 181.6 | **1.2×** |
+| detection: COCO evaluation (1000 images) | pycocotools | 817.3 | 983.7 | **1.2×** |
+| multiple testing: Hochberg (n p-values) | statsmodels | 82.0 | 108.0 | **1.3×** |
+| regression: MAE, MSE, RMSE, R² via evaluate() | scikit-learn | 23.1 | 10.8 | 0.47× |
+| segmentation: Dice and IoU per class (n = pixels) | scikit-learn | 43.0 | 290.0 | **6.8×** |
+| segmentation: Hausdorff distance (50 images) | SciPy | 27.0 | 20.2 | 0.75× |
+| statistics: Cramér's V (5×5 table) | SciPy | 0.5 | 0.4 | 0.88× |
+| statistics: Mann–Whitney U | SciPy | 349.7 | 350.2 | **1.0×** |
+| statistics: Welch t-test | SciPy | 14.5 | 7.2 | 0.49× |
 
 `evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where most
 of the speed-up comes from. Hypothesis tests use SciPy underneath, so they match its speed at best; rows
@@ -225,6 +273,13 @@ Kruskal–Wallis, Friedman, Shapiro–Wilk, χ², Fisher's exact; effect sizes (
 delta, Cramér's V); multiple-testing corrections (Bonferroni, Holm, Hochberg, Benjamini–Hochberg,
 Benjamini–Yekutieli).
 
+**Segmentation** (2-D and 3-D label masks; `ignore_index`; dataset or per-image aggregation): Dice, IoU,
+mIoU, pixel accuracy, mean pixel accuracy, Boundary IoU, Hausdorff distance and HD95, average symmetric
+surface distance (with pixel spacing), confusion matrix and a full report.
+
+**Object detection**: box IoU (xyxy, xywh, cxcywh), COCO mAP@[.50:.95], mAP@.50, mAP@.75, mAP and mAR by
+object size, AP per class with COCO or VOC interpolation, precision-recall curves, COCO file import.
+
 **Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
 MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
 loss, Huber loss, relative absolute error, relative squared error.

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/pyproject.toml

@@ -12,12 +12,13 @@ license-files = ["LICENSE"]
 requires-python = ">=3.9"
 authors = [{ name = "Manoj Kumar C S" }, { name = "Nikhil D Bharadwaj" }]
 maintainers = [{ name = "Manoj Kumar C S" }, { name = "Nikhil D Bharadwaj" }]
-keywords = ["evaluation", "metrics", "clinical", "diagnostic accuracy", "calibration", "statistical tests", "machine learning", "statistics", "classification", "regression", "reproducibility"]
+keywords = ["evaluation", "metrics", "segmentation", "object detection", "mAP", "clinical", "diagnostic accuracy", "calibration", "statistical tests", "machine learning", "statistics", "classification", "regression", "reproducibility"]
 classifiers = [
   "Development Status :: 5 - Production/Stable",
   "Intended Audience :: Science/Research",
   "Intended Audience :: Healthcare Industry",
   "Topic :: Scientific/Engineering :: Medical Science Apps.",
+  "Topic :: Scientific/Engineering :: Image Recognition",
   "Intended Audience :: Developers",
   "Operating System :: OS Independent",
   "Programming Language :: Python :: 3",
@@ -36,14 +37,17 @@ dependencies = ["numpy>=1.22", "scipy>=1.8", "pandas>=1.4"]
 
 [project.optional-dependencies]
 plot = ["matplotlib>=3.5"]
-all = ["matplotlib>=3.5"]
+vision = ["pillow>=9"]
+all = ["matplotlib>=3.5", "pillow>=9"]
 dev = [
   "pytest>=7",
   "pytest-cov>=4",
   "hypothesis>=6.80",
-  "scikit-learn>=1.2",
+  "scikit-learn>=1.4",
   "statsmodels>=0.13",
   "matplotlib>=3.5",
+  "pillow>=9",
+  "pycocotools>=2.0.7",
   "ruff>=0.6",
   "mypy>=1.10",
   "pandas-stubs",

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/src/evalsuite/__init__.py

@@ -5,7 +5,7 @@
 >>> print(result.summary())  # doctest: +SKIP
 """
 
-from . import calibration, classification, clinical, plot, regression, stats
+from . import calibration, classification, clinical, plot, regression, stats, vision
 from .api import evaluate
 from .calibration import (
     CalibrationReport,
@@ -111,8 +111,49 @@ from .stats import (
     wilcoxon_test,
 )
 from .version import __version__
+from .vision import (
+    DetectionReport,
+    SegmentationReport,
+    average_precision_detection,
+    average_surface_distance,
+    boundary_iou,
+    box_iou,
+    detection_pr_curve,
+    detection_report,
+    dice,
+    from_coco,
+    hausdorff_distance,
+    iou,
+    mean_average_precision,
+    mean_pixel_accuracy,
+    miou,
+    per_image_scores,
+    pixel_accuracy,
+    segmentation_confusion,
+    segmentation_report,
+)
 
 __all__ = [
+    "SegmentationReport",
+    "segmentation_report",
+    "detection_pr_curve",
+    "vision",
+    "DetectionReport",
+    "average_precision_detection",
+    "average_surface_distance",
+    "boundary_iou",
+    "box_iou",
+    "detection_report",
+    "dice",
+    "from_coco",
+    "hausdorff_distance",
+    "iou",
+    "mean_average_precision",
+    "mean_pixel_accuracy",
+    "miou",
+    "per_image_scores",
+    "pixel_accuracy",
+    "segmentation_confusion",
     "CalibrationReport",
     "calibration_report",
     "calibration",

{evalsuite_python-0.2.1 → evalsuite_python-0.3.1}/src/evalsuite/api.py

@@ -149,6 +149,16 @@ def evaluate(
     >>> round(r["accuracy"], 2)
     0.75
     """
+    if isinstance(y_true, (list, tuple)) and y_true and isinstance(y_true[0], dict):
+        raise UnsupportedTaskError(
+            "y_true looks like object detection annotations (one dict per image); use "
+            "evalsuite.detection_report(y_true, y_pred) or evalsuite.mean_average_precision(...)."
+        )
+    if np.ndim(y_true) >= 3:
+        raise UnsupportedTaskError(
+            "y_true has 3 or more dimensions, which looks like segmentation masks (images first); use "
+            "evalsuite.segmentation_report(y_true, y_pred) or evalsuite.dice / evalsuite.iou."
+        )
     task = task or _infer_task(y_true, y_prob)
     if task == "classification":
         return _evaluate_classification(