evalsuite-python 0.2.0__tar.gz → 0.3.0__tar.gz

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/CHANGELOG.md

@@ -6,6 +6,49 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-10-09
+
+Computer vision (the v0.3.0 roadmap), with comparison, plots, reporting and benchmarks extended to it.
+
+### Added
+- Segmentation: `dice`, `iou`, `miou`, `pixel_accuracy`, `mean_pixel_accuracy`, `boundary_iou`,
+  `hausdorff_distance` (HD and HD95, anisotropic `spacing`), `average_surface_distance`,
+  `segmentation_confusion`, `per_image_scores` and `segmentation_report`. 2-D and 3-D masks, lists of
+  differently sized masks, `ignore_index`, `aggregate="dataset"|"image"`, undefined classes reported as NaN.
+- Object detection: `box_iou`, `detection_report` (the 12 COCO numbers plus AP per class),
+  `mean_average_precision`, `average_precision_detection` (COCO or VOC interpolation), `detection_pr_curve`,
+  `from_coco`. Matches pycocotools exactly, including crowd regions, area ranges and detection limits.
+- Comparison for vision: `compare`, `bootstrap_ci` and `paired_bootstrap_test` resample images for
+  segmentation masks and per-image detections.
+- Plots: `es.plot.segmentation` (prediction fill vs truth outline), `es.plot.per_class` (any per-class
+  result, a segmentation report or a detection report), `es.plot.detection_pr`.
+- CLI: `evalsuite segmentation` (.npy/.npz or a folder of mask images) and `evalsuite detection` (COCO JSON).
+- Benchmarks: `--suite vision` against scikit-learn, SciPy and pycocotools.
+- `vision` extra (Pillow, for reading mask image folders); `CITATION.cff`.
+
+### Changed
+- `evaluate()` points segmentation masks and detection annotations to the right functions instead of
+  failing with a shape error.
+- CI and release workflows use the Node 24 versions of the GitHub actions.
+
+### Fixed
+- Calibration slope and intercept no longer emit an overflow warning before reporting a perfectly
+  separated outcome.
+- The CLI no longer fails on consoles that cannot print Unicode (Windows code pages).
+
+## [0.2.1] - 2026-10-09
+
+### Added
+- Benchmarks for the v0.2.0 functions (`evalsuite benchmark --suite clinical`): diagnostic metrics and
+  report, calibration slope and intercept, decision curves, t-test, Mann–Whitney, Cramér's V and Hochberg,
+  each against its reference (scikit-learn, statsmodels, SciPy or the textbook NumPy loop). Benchmark rows
+  now name their reference library.
+
+### Changed
+- Faster label handling: integer class labels are found with one marking pass instead of a sort
+  (8 label metrics at 1M samples: 34× faster than scikit-learn, up from 10×; macro F1 5.8×, up from 1.6×).
+- Decision curves sort the risks once and use cumulative sums (O((n + k) log n), no n × k matrix).
+
 ## [0.2.0] - 2026-10-08
 
 Clinical and statistical evaluation (the v0.2.0 roadmap).

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: evalsuite-python
-Version: 0.2.0
+Version: 0.3.0
 Summary: A unified Python framework for machine-learning, clinical, statistical, segmentation, object-detection, uncertainty, and model evaluation.
 Project-URL: Homepage, https://evalsuite-nine.vercel.app
 Project-URL: Documentation, https://evalsuite-nine.vercel.app/docs
@@ -11,7 +11,7 @@ Author: Manoj Kumar C S, Nikhil D Bharadwaj
 Maintainer: Manoj Kumar C S, Nikhil D Bharadwaj
 License-Expression: MIT
 License-File: LICENSE
-Keywords: calibration,classification,clinical,diagnostic accuracy,evaluation,machine learning,metrics,regression,reproducibility,statistical tests,statistics
+Keywords: calibration,classification,clinical,diagnostic accuracy,evaluation,mAP,machine learning,metrics,object detection,regression,reproducibility,segmentation,statistical tests,statistics
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Healthcare Industry
@@ -27,6 +27,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Scientific/Engineering
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
 Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
 Classifier: Typing :: Typed
 Requires-Python: >=3.9
@@ -35,13 +36,16 @@ Requires-Dist: pandas>=1.4
 Requires-Dist: scipy>=1.8
 Provides-Extra: all
 Requires-Dist: matplotlib>=3.5; extra == 'all'
+Requires-Dist: pillow>=9; extra == 'all'
 Provides-Extra: dev
 Requires-Dist: build; extra == 'dev'
 Requires-Dist: hypothesis>=6.80; extra == 'dev'
 Requires-Dist: matplotlib>=3.5; extra == 'dev'
 Requires-Dist: mypy>=1.10; extra == 'dev'
 Requires-Dist: pandas-stubs; extra == 'dev'
+Requires-Dist: pillow>=9; extra == 'dev'
 Requires-Dist: pip-audit; extra == 'dev'
+Requires-Dist: pycocotools>=2.0.7; extra == 'dev'
 Requires-Dist: pytest-cov>=4; extra == 'dev'
 Requires-Dist: pytest>=7; extra == 'dev'
 Requires-Dist: ruff>=0.6; extra == 'dev'
@@ -50,6 +54,8 @@ Requires-Dist: statsmodels>=0.13; extra == 'dev'
 Requires-Dist: twine; extra == 'dev'
 Provides-Extra: plot
 Requires-Dist: matplotlib>=3.5; extra == 'plot'
+Provides-Extra: vision
+Requires-Dist: pillow>=9; extra == 'vision'
 Description-Content-Type: text/markdown
 
 # EvalSuite
@@ -61,10 +67,10 @@ Description-Content-Type: text/markdown
 
 **Unified, reproducible evaluation for machine learning and research.**
 
-EvalSuite brings classification, regression, clinical and statistical evaluation (with segmentation and
-object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+EvalSuite brings classification, regression, clinical, statistical, segmentation and object-detection
+evaluation into one consistent, validated, documented framework.
 
-> **Status: stable (0.2.0).** Every item on the 0.1.0 and 0.2.0 roadmaps is implemented and verified.
+> **Status: stable (0.3.0).** Every item on the 0.1.0, 0.2.0 and 0.3.0 roadmaps is implemented and verified.
 
 ## Installation
 
@@ -181,6 +187,45 @@ es.adjust_pvalues(p_values, method="hochberg")  # also holm, bonferroni, bh, by
 Every test returns a `TestResult` with the statistic, p-value and an effect size, computed with SciPy and
 checked against SciPy and statsmodels in the test suite.
 
+## Segmentation
+
+```python
+# label masks: one 2-D image, or images on the first axis (N, H, W) / (N, D, H, W), or a list of masks
+es.dice(y_true, y_pred)  # macro over classes, pixel counts summed over the dataset
+es.iou(y_true, y_pred, average=None)  # per class; classes absent from both masks are NaN, not 0
+es.miou(y_true, y_pred, ignore_index=255)
+es.dice(y_true, y_pred, aggregate="image")  # mean of per-image scores (medical imaging convention)
+es.boundary_iou(y_true, y_pred)  # Cheng et al. 2021
+es.hausdorff_distance(y_true, y_pred, percentile=95, spacing=(0.8, 0.8))  # HD95 in mm
+
+report = es.segmentation_report(y_true, y_pred, class_names={0: "background", 1: "liver"})
+print(report)  # mIoU, Dice, pixel accuracy, Boundary IoU, HD95, ASSD + per-class table
+es.plot.segmentation(image, y_true[0], y_pred[0])  # prediction fill, truth outline
+es.plot.per_class(report, metric="iou")  # per-class bars (also takes a detection report)
+```
+
+## Object detection
+
+```python
+y_true = [{"boxes": [[x1, y1, x2, y2], ...], "labels": [3, ...]}, ...]  # one dict per image
+y_pred = [{"boxes": [...], "labels": [...], "scores": [...]}, ...]
+
+report = es.detection_report(y_true, y_pred)  # the 12 COCO numbers + AP per class
+report["map"], report["map_50"], report["mar_100"]
+es.mean_average_precision(y_true, y_pred, iou_threshold=0.5)  # mAP@.50
+es.average_precision_detection(y_true, y_pred, interpolation="voc")  # per class, VOC-style
+es.box_iou(boxes_a, boxes_b, box_format="xywh")
+y_true, y_pred = es.from_coco("instances_val.json", "detections.json")
+es.plot.detection_pr(y_true, y_pred)
+```
+
+The COCO protocol (crowd regions, area ranges, max detections, 101-point interpolation) matches
+`pycocotools` to the last digit in the test suite.
+
+Models can be compared over the same images with intervals and paired tests, as for every other task:
+`es.compare(y_true, {"unet": masks_a, "deeplab": masks_b})` resamples images; for detection it compares
+mAP.
+
 ## Classification report
 
 ```python
@@ -225,6 +270,8 @@ evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
 evalsuite diagnostic predictions.csv --y-true label --y-pred pred   # sensitivity, LR+, DOR... with CIs
 evalsuite calibration predictions.csv --y-true label --y-prob prob  # slope, intercept, ECE, HL
 evalsuite plot decision predictions.csv --y-true label --y-prob prob -o dca.png
+evalsuite segmentation true_masks.npy pred_masks.npy --ignore-index 255 --plot per_class.png
+evalsuite detection instances_val.json detections.json --plot pr_curves.png
 evalsuite metrics --category clinical
 evalsuite info classification.mcc
 evalsuite benchmark --quick
@@ -235,24 +282,30 @@ Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` o
 
 ## Performance
 
-Benchmarked against scikit-learn on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
-scikit-learn 1.9, Linux x86_64). Every result agrees with scikit-learn to floating-point rounding
-(largest difference 1.1e-16).
-
-| Case | n | EvalSuite (ms) | scikit-learn (ms) | Speed-up | Peak memory EvalSuite / sklearn (MiB) |
-| --- | ---: | ---: | ---: | ---: | ---: |
-| 8 binary label metrics via `evaluate()` | 1,000 | 0.38 | 11.70 | **31.2×** | 0.04 / 0.05 |
-| 8 binary label metrics via `evaluate()` | 100,000 | 10.9 | 116.9 | **10.7×** | 3.2 / 3.1 |
-| 8 binary label metrics via `evaluate()` | 1,000,000 | 108.8 | 1043.6 | **9.6×** | 31.5 / 30.5 |
-| macro F1, 10 classes | 1,000,000 | 88.4 | 139.3 | **1.58×** | 30.5 / 21.8 |
-| ROC AUC, binary | 1,000,000 | 247.4 | 352.5 | **1.42×** | 91.6 / 76.3 |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000 | 0.12 | 0.90 | **7.2×** | 0.03 / 0.02 |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | 29.3 | 20.1 | 0.69× | 22.9 / 15.3 |
-
-`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where the
-speed-up comes from. Large regression arrays are slower because EvalSuite checks every value for NaN,
-infinity, shape and dtype before computing. Reproduce on your machine with `evalsuite benchmark`; full
-table and notes in
+Benchmarked against reference implementations on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
+Linux x86_64). Every result agrees with the reference to floating-point rounding (largest difference
+1.4e-14).
+
+| Case | n | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
+| --- | ---: | --- | ---: | ---: | ---: |
+| 8 binary label metrics via `evaluate()` | 1,000,000 | scikit-learn | 30.2 | 1020.2 | **33.8×** |
+| macro F1, 10 classes | 1,000,000 | scikit-learn | 22.4 | 128.7 | **5.8×** |
+| ROC AUC, binary | 1,000,000 | scikit-learn | 173.2 | 300.6 | **1.7×** |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | scikit-learn | 19.1 | 9.7 | 0.51× |
+| sensitivity, specificity, LR+, LR− | 1,000,000 | scikit-learn | 66.2 | 392.8 | **5.9×** |
+| calibration slope and intercept | 1,000,000 | statsmodels | 178.0 | 1014.8 | **5.7×** |
+| decision curve, 99 thresholds | 1,000,000 | NumPy loop | 155.2 | 174.3 | **1.1×** |
+| diagnostic report (7 CIs) | 1,000,000 | statsmodels | 16.6 | 4.6 | 0.28× |
+| Welch t-test | 1,000,000 | SciPy | 14.5 | 7.3 | 0.51× |
+| Hochberg correction | 1,000,000 | statsmodels | 75.4 | 81.9 | **1.1×** |
+| segmentation Dice and IoU per class | 1,000,000 px | scikit-learn | 40.2 | 282.0 | **7.0×** |
+| COCO detection evaluation (12 numbers) | 1,000 images | pycocotools | 967.4 | 980.6 | **1.0×** |
+| Hausdorff distance | 50 images | SciPy | 26.2 | 20.8 | 0.79× |
+
+`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where most
+of the speed-up comes from. Hypothesis tests use SciPy underneath, so they match its speed at best; rows
+below 1× pay for input validation and the extra intervals and effect sizes EvalSuite reports. Reproduce on
+your machine with `evalsuite benchmark`; full table (1k, 100k and 1M samples, peak memory) and notes in
 [BENCHMARKS.md](https://github.com/mkcs28/evalsuite-python/blob/main/BENCHMARKS.md).
 
 ## Metrics in this release
@@ -276,6 +329,13 @@ Kruskal–Wallis, Friedman, Shapiro–Wilk, χ², Fisher's exact; effect sizes (
 delta, Cramér's V); multiple-testing corrections (Bonferroni, Holm, Hochberg, Benjamini–Hochberg,
 Benjamini–Yekutieli).
 
+**Segmentation** (2-D and 3-D label masks; `ignore_index`; dataset or per-image aggregation): Dice, IoU,
+mIoU, pixel accuracy, mean pixel accuracy, Boundary IoU, Hausdorff distance and HD95, average symmetric
+surface distance (with pixel spacing), confusion matrix and a full report.
+
+**Object detection**: box IoU (xyxy, xywh, cxcywh), COCO mAP@[.50:.95], mAP@.50, mAP@.75, mAP and mAR by
+object size, AP per class with COCO or VOC interpolation, precision-recall curves, COCO file import.
+
 **Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
 MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
 loss, Huber loss, relative absolute error, relative squared error.

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/README.md

@@ -7,10 +7,10 @@
 
 **Unified, reproducible evaluation for machine learning and research.**
 
-EvalSuite brings classification, regression, clinical and statistical evaluation (with segmentation and
-object-detection evaluation on the roadmap) into one consistent, validated, documented framework.
+EvalSuite brings classification, regression, clinical, statistical, segmentation and object-detection
+evaluation into one consistent, validated, documented framework.
 
-> **Status: stable (0.2.0).** Every item on the 0.1.0 and 0.2.0 roadmaps is implemented and verified.
+> **Status: stable (0.3.0).** Every item on the 0.1.0, 0.2.0 and 0.3.0 roadmaps is implemented and verified.
 
 ## Installation
 
@@ -127,6 +127,45 @@ es.adjust_pvalues(p_values, method="hochberg")  # also holm, bonferroni, bh, by
 Every test returns a `TestResult` with the statistic, p-value and an effect size, computed with SciPy and
 checked against SciPy and statsmodels in the test suite.
 
+## Segmentation
+
+```python
+# label masks: one 2-D image, or images on the first axis (N, H, W) / (N, D, H, W), or a list of masks
+es.dice(y_true, y_pred)  # macro over classes, pixel counts summed over the dataset
+es.iou(y_true, y_pred, average=None)  # per class; classes absent from both masks are NaN, not 0
+es.miou(y_true, y_pred, ignore_index=255)
+es.dice(y_true, y_pred, aggregate="image")  # mean of per-image scores (medical imaging convention)
+es.boundary_iou(y_true, y_pred)  # Cheng et al. 2021
+es.hausdorff_distance(y_true, y_pred, percentile=95, spacing=(0.8, 0.8))  # HD95 in mm
+
+report = es.segmentation_report(y_true, y_pred, class_names={0: "background", 1: "liver"})
+print(report)  # mIoU, Dice, pixel accuracy, Boundary IoU, HD95, ASSD + per-class table
+es.plot.segmentation(image, y_true[0], y_pred[0])  # prediction fill, truth outline
+es.plot.per_class(report, metric="iou")  # per-class bars (also takes a detection report)
+```
+
+## Object detection
+
+```python
+y_true = [{"boxes": [[x1, y1, x2, y2], ...], "labels": [3, ...]}, ...]  # one dict per image
+y_pred = [{"boxes": [...], "labels": [...], "scores": [...]}, ...]
+
+report = es.detection_report(y_true, y_pred)  # the 12 COCO numbers + AP per class
+report["map"], report["map_50"], report["mar_100"]
+es.mean_average_precision(y_true, y_pred, iou_threshold=0.5)  # mAP@.50
+es.average_precision_detection(y_true, y_pred, interpolation="voc")  # per class, VOC-style
+es.box_iou(boxes_a, boxes_b, box_format="xywh")
+y_true, y_pred = es.from_coco("instances_val.json", "detections.json")
+es.plot.detection_pr(y_true, y_pred)
+```
+
+The COCO protocol (crowd regions, area ranges, max detections, 101-point interpolation) matches
+`pycocotools` to the last digit in the test suite.
+
+Models can be compared over the same images with intervals and paired tests, as for every other task:
+`es.compare(y_true, {"unet": masks_a, "deeplab": masks_b})` resamples images; for detection it compares
+mAP.
+
 ## Classification report
 
 ```python
@@ -171,6 +210,8 @@ evalsuite plot roc predictions.csv --y-true label --y-prob prob -o roc.png
 evalsuite diagnostic predictions.csv --y-true label --y-pred pred   # sensitivity, LR+, DOR... with CIs
 evalsuite calibration predictions.csv --y-true label --y-prob prob  # slope, intercept, ECE, HL
 evalsuite plot decision predictions.csv --y-true label --y-prob prob -o dca.png
+evalsuite segmentation true_masks.npy pred_masks.npy --ignore-index 255 --plot per_class.png
+evalsuite detection instances_val.json detections.json --plot pr_curves.png
 evalsuite metrics --category clinical
 evalsuite info classification.mcc
 evalsuite benchmark --quick
@@ -181,24 +222,30 @@ Input files can be CSV, TSV, Parquet or JSON. Output format follows `--format` o
 
 ## Performance
 
-Benchmarked against scikit-learn on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
-scikit-learn 1.9, Linux x86_64). Every result agrees with scikit-learn to floating-point rounding
-(largest difference 1.1e-16).
-
-| Case | n | EvalSuite (ms) | scikit-learn (ms) | Speed-up | Peak memory EvalSuite / sklearn (MiB) |
-| --- | ---: | ---: | ---: | ---: | ---: |
-| 8 binary label metrics via `evaluate()` | 1,000 | 0.38 | 11.70 | **31.2×** | 0.04 / 0.05 |
-| 8 binary label metrics via `evaluate()` | 100,000 | 10.9 | 116.9 | **10.7×** | 3.2 / 3.1 |
-| 8 binary label metrics via `evaluate()` | 1,000,000 | 108.8 | 1043.6 | **9.6×** | 31.5 / 30.5 |
-| macro F1, 10 classes | 1,000,000 | 88.4 | 139.3 | **1.58×** | 30.5 / 21.8 |
-| ROC AUC, binary | 1,000,000 | 247.4 | 352.5 | **1.42×** | 91.6 / 76.3 |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000 | 0.12 | 0.90 | **7.2×** | 0.03 / 0.02 |
-| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | 29.3 | 20.1 | 0.69× | 22.9 / 15.3 |
-
-`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where the
-speed-up comes from. Large regression arrays are slower because EvalSuite checks every value for NaN,
-infinity, shape and dtype before computing. Reproduce on your machine with `evalsuite benchmark`; full
-table and notes in
+Benchmarked against reference implementations on the same data (fastest of 5 runs; Python 3.12, NumPy 2.5,
+Linux x86_64). Every result agrees with the reference to floating-point rounding (largest difference
+1.4e-14).
+
+| Case | n | Reference | EvalSuite (ms) | Reference (ms) | Speed-up |
+| --- | ---: | --- | ---: | ---: | ---: |
+| 8 binary label metrics via `evaluate()` | 1,000,000 | scikit-learn | 30.2 | 1020.2 | **33.8×** |
+| macro F1, 10 classes | 1,000,000 | scikit-learn | 22.4 | 128.7 | **5.8×** |
+| ROC AUC, binary | 1,000,000 | scikit-learn | 173.2 | 300.6 | **1.7×** |
+| MAE, MSE, RMSE, R² via `evaluate()` | 1,000,000 | scikit-learn | 19.1 | 9.7 | 0.51× |
+| sensitivity, specificity, LR+, LR− | 1,000,000 | scikit-learn | 66.2 | 392.8 | **5.9×** |
+| calibration slope and intercept | 1,000,000 | statsmodels | 178.0 | 1014.8 | **5.7×** |
+| decision curve, 99 thresholds | 1,000,000 | NumPy loop | 155.2 | 174.3 | **1.1×** |
+| diagnostic report (7 CIs) | 1,000,000 | statsmodels | 16.6 | 4.6 | 0.28× |
+| Welch t-test | 1,000,000 | SciPy | 14.5 | 7.3 | 0.51× |
+| Hochberg correction | 1,000,000 | statsmodels | 75.4 | 81.9 | **1.1×** |
+| segmentation Dice and IoU per class | 1,000,000 px | scikit-learn | 40.2 | 282.0 | **7.0×** |
+| COCO detection evaluation (12 numbers) | 1,000 images | pycocotools | 967.4 | 980.6 | **1.0×** |
+| Hausdorff distance | 50 images | SciPy | 26.2 | 20.8 | 0.79× |
+
+`evaluate()` validates inputs once and builds the confusion matrix once for all metrics, which is where most
+of the speed-up comes from. Hypothesis tests use SciPy underneath, so they match its speed at best; rows
+below 1× pay for input validation and the extra intervals and effect sizes EvalSuite reports. Reproduce on
+your machine with `evalsuite benchmark`; full table (1k, 100k and 1M samples, peak memory) and notes in
 [BENCHMARKS.md](https://github.com/mkcs28/evalsuite-python/blob/main/BENCHMARKS.md).
 
 ## Metrics in this release
@@ -222,6 +269,13 @@ Kruskal–Wallis, Friedman, Shapiro–Wilk, χ², Fisher's exact; effect sizes (
 delta, Cramér's V); multiple-testing corrections (Bonferroni, Holm, Hochberg, Benjamini–Hochberg,
 Benjamini–Yekutieli).
 
+**Segmentation** (2-D and 3-D label masks; `ignore_index`; dataset or per-image aggregation): Dice, IoU,
+mIoU, pixel accuracy, mean pixel accuracy, Boundary IoU, Hausdorff distance and HD95, average symmetric
+surface distance (with pixel spacing), confusion matrix and a full report.
+
+**Object detection**: box IoU (xyxy, xywh, cxcywh), COCO mAP@[.50:.95], mAP@.50, mAP@.75, mAP and mAR by
+object size, AP per class with COCO or VOC interpolation, precision-recall curves, COCO file import.
+
 **Regression** (single and multi-output; sample weights): MAE, MSE, RMSE, R², adjusted R², MAPE, sMAPE,
 MSLE, RMSLE, median absolute error, explained variance, max error, mean bias error, quantile (pinball)
 loss, Huber loss, relative absolute error, relative squared error.

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/pyproject.toml

@@ -12,12 +12,13 @@ license-files = ["LICENSE"]
 requires-python = ">=3.9"
 authors = [{ name = "Manoj Kumar C S" }, { name = "Nikhil D Bharadwaj" }]
 maintainers = [{ name = "Manoj Kumar C S" }, { name = "Nikhil D Bharadwaj" }]
-keywords = ["evaluation", "metrics", "clinical", "diagnostic accuracy", "calibration", "statistical tests", "machine learning", "statistics", "classification", "regression", "reproducibility"]
+keywords = ["evaluation", "metrics", "segmentation", "object detection", "mAP", "clinical", "diagnostic accuracy", "calibration", "statistical tests", "machine learning", "statistics", "classification", "regression", "reproducibility"]
 classifiers = [
   "Development Status :: 5 - Production/Stable",
   "Intended Audience :: Science/Research",
   "Intended Audience :: Healthcare Industry",
   "Topic :: Scientific/Engineering :: Medical Science Apps.",
+  "Topic :: Scientific/Engineering :: Image Recognition",
   "Intended Audience :: Developers",
   "Operating System :: OS Independent",
   "Programming Language :: Python :: 3",
@@ -36,7 +37,8 @@ dependencies = ["numpy>=1.22", "scipy>=1.8", "pandas>=1.4"]
 
 [project.optional-dependencies]
 plot = ["matplotlib>=3.5"]
-all = ["matplotlib>=3.5"]
+vision = ["pillow>=9"]
+all = ["matplotlib>=3.5", "pillow>=9"]
 dev = [
   "pytest>=7",
   "pytest-cov>=4",
@@ -44,6 +46,8 @@ dev = [
   "scikit-learn>=1.2",
   "statsmodels>=0.13",
   "matplotlib>=3.5",
+  "pillow>=9",
+  "pycocotools>=2.0.7",
   "ruff>=0.6",
   "mypy>=1.10",
   "pandas-stubs",

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/src/evalsuite/__init__.py

@@ -5,7 +5,7 @@
 >>> print(result.summary())  # doctest: +SKIP
 """
 
-from . import calibration, classification, clinical, plot, regression, stats
+from . import calibration, classification, clinical, plot, regression, stats, vision
 from .api import evaluate
 from .calibration import (
     CalibrationReport,
@@ -111,8 +111,49 @@ from .stats import (
     wilcoxon_test,
 )
 from .version import __version__
+from .vision import (
+    DetectionReport,
+    SegmentationReport,
+    average_precision_detection,
+    average_surface_distance,
+    boundary_iou,
+    box_iou,
+    detection_pr_curve,
+    detection_report,
+    dice,
+    from_coco,
+    hausdorff_distance,
+    iou,
+    mean_average_precision,
+    mean_pixel_accuracy,
+    miou,
+    per_image_scores,
+    pixel_accuracy,
+    segmentation_confusion,
+    segmentation_report,
+)
 
 __all__ = [
+    "SegmentationReport",
+    "segmentation_report",
+    "detection_pr_curve",
+    "vision",
+    "DetectionReport",
+    "average_precision_detection",
+    "average_surface_distance",
+    "boundary_iou",
+    "box_iou",
+    "detection_report",
+    "dice",
+    "from_coco",
+    "hausdorff_distance",
+    "iou",
+    "mean_average_precision",
+    "mean_pixel_accuracy",
+    "miou",
+    "per_image_scores",
+    "pixel_accuracy",
+    "segmentation_confusion",
     "CalibrationReport",
     "calibration_report",
     "calibration",

{evalsuite_python-0.2.0 → evalsuite_python-0.3.0}/src/evalsuite/api.py

@@ -149,6 +149,16 @@ def evaluate(
     >>> round(r["accuracy"], 2)
     0.75
     """
+    if isinstance(y_true, (list, tuple)) and y_true and isinstance(y_true[0], dict):
+        raise UnsupportedTaskError(
+            "y_true looks like object detection annotations (one dict per image); use "
+            "evalsuite.detection_report(y_true, y_pred) or evalsuite.mean_average_precision(...)."
+        )
+    if np.ndim(y_true) >= 3:
+        raise UnsupportedTaskError(
+            "y_true has 3 or more dimensions, which looks like segmentation masks (images first); use "
+            "evalsuite.segmentation_report(y_true, y_pred) or evalsuite.dice / evalsuite.iou."
+        )
     task = task or _infer_task(y_true, y_prob)
     if task == "classification":
         return _evaluate_classification(