faster-coco-eval 1.7.2__tar.gz → 1.8.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- faster_coco_eval-1.8.0/MANIFEST.in +5 -0
- {faster_coco_eval-1.7.2/faster_coco_eval.egg-info → faster_coco_eval-1.8.0}/PKG-INFO +30 -28
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/README.md +27 -22
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/coco_eval/cocoeval.cpp +388 -140
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/coco_eval/dataset.cpp +6 -5
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/coco_eval/dataset.h +2 -2
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/mask_api/src/mask.cpp +212 -90
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/mask_api/src/mask.h +32 -2
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/mask_api/src/rle.cpp +34 -10
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/core/coco.py +126 -84
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/core/cocoeval.py +111 -81
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/core/faster_eval_api.py +115 -23
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/core/mask.py +12 -19
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/curves.py +40 -21
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/display.py +14 -13
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/draw.py +49 -34
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/extra.py +21 -22
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/utils.py +4 -1
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/utils/pytorch/coco_dataset.py +7 -10
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/utils/pytorch/coco_eval.py +19 -22
- faster_coco_eval-1.8.0/faster_coco_eval/version.py +2 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0/faster_coco_eval.egg-info}/PKG-INFO +30 -28
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval.egg-info/SOURCES.txt +13 -1
- faster_coco_eval-1.8.0/history.md +228 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/pyproject.toml +16 -4
- faster_coco_eval-1.8.0/requirements/docs.txt +6 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/setup.py +5 -6
- faster_coco_eval-1.8.0/tests/test_annotation_id_zero.py +56 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_basic.py +40 -17
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_boundary.py +5 -1
- faster_coco_eval-1.8.0/tests/test_coco_api_and_evaluator_regressions.py +562 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_coco_metric.py +3 -1
- faster_coco_eval-1.8.0/tests/test_compute_oks.py +69 -0
- faster_coco_eval-1.8.0/tests/test_core_low_regressions.py +105 -0
- faster_coco_eval-1.8.0/tests/test_cpp_safety_regressions.py +108 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_crowdpose.py +4 -1
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_dataset.py +226 -0
- faster_coco_eval-1.8.0/tests/test_display_matrix_normalization.py +42 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_extensive_pycocotools_comparison.py +190 -168
- faster_coco_eval-1.8.0/tests/test_extra_curves.py +58 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_extra_draw.py +36 -8
- faster_coco_eval-1.8.0/tests/test_extra_regressions.py +92 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_extra_utils.py +42 -6
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_keypoints.py +14 -4
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_lvis_metric.py +2 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_mask_api.py +249 -0
- faster_coco_eval-1.8.0/tests/test_orphan_annotations.py +88 -0
- faster_coco_eval-1.8.0/tests/test_pytorch_coco_dataset.py +56 -0
- faster_coco_eval-1.8.0/tests/test_pytorch_coco_eval.py +68 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_simple_extra.py +93 -4
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_torchmetrics.py +19 -8
- faster_coco_eval-1.8.0/tests/test_world_evaluator.py +124 -0
- faster_coco_eval-1.7.2/MANIFEST.in +0 -1
- faster_coco_eval-1.7.2/faster_coco_eval/version.py +0 -2
- faster_coco_eval-1.7.2/requirements/docs.txt +0 -8
- faster_coco_eval-1.7.2/tests/test_world_evalutor.py +0 -84
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/LICENSE +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/coco_eval/cocoeval.h +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/coco_eval/types.h +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/faster_eval_api/faster_eval_api.cpp +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/csrc/mask_api/mask_api.cpp +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/__init__.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/core/__init__.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/extra/__init__.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/utils/__init__.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval/utils/pytorch/__init__.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval.egg-info/dependency_links.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval.egg-info/requires.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/faster_coco_eval.egg-info/top_level.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/requirements/extra.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/requirements/runtime.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/requirements/tests.txt +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/setup.cfg +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_cocoapi_fake_data.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_init_pycocotools.py +0 -0
- {faster_coco_eval-1.7.2 → faster_coco_eval-1.8.0}/tests/test_ranges.py +0 -0
|
@@ -1,22 +1,19 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: faster-coco-eval
|
|
3
|
-
Version: 1.
|
|
3
|
+
Version: 1.8.0
|
|
4
4
|
Summary: Faster interpretation of the original COCOEval
|
|
5
5
|
Home-page: https://github.com/MiXaiLL76/faster_coco_eval
|
|
6
|
-
Author: MiXaiLL76
|
|
6
|
+
Author: MiXaiLL76,Borda
|
|
7
7
|
Author-email: mike.milos@yandex.ru
|
|
8
8
|
Classifier: Development Status :: 4 - Beta
|
|
9
9
|
Classifier: Operating System :: OS Independent
|
|
10
10
|
Classifier: Programming Language :: Python :: 3
|
|
11
|
-
Classifier: Programming Language :: Python :: 3.7
|
|
12
|
-
Classifier: Programming Language :: Python :: 3.8
|
|
13
|
-
Classifier: Programming Language :: Python :: 3.9
|
|
14
11
|
Classifier: Programming Language :: Python :: 3.10
|
|
15
12
|
Classifier: Programming Language :: Python :: 3.11
|
|
16
13
|
Classifier: Programming Language :: Python :: 3.12
|
|
17
14
|
Classifier: Programming Language :: Python :: 3.13
|
|
18
15
|
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
19
|
-
Requires-Python: >=3.
|
|
16
|
+
Requires-Python: >=3.10
|
|
20
17
|
Description-Content-Type: text/markdown
|
|
21
18
|
License-File: LICENSE
|
|
22
19
|
Requires-Dist: numpy
|
|
@@ -63,20 +60,20 @@ Dynamic: summary
|
|
|
63
60
|
| ------------------------------ | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
64
61
|
| **Support & Development** | Outdated and not actively maintained. Issues and incompatibilities arise with new releases. | Actively maintained, continuously evolving, and regularly updated with new features and bug fixes. |
|
|
65
62
|
| **Transparency & Reliability** | Lacks comprehensive testing, making updates risky and results less predictable. | Emphasizes extensive test coverage and code quality, ensuring trustworthy and reliable results. |
|
|
66
|
-
| **Performance** | Significantly slower, especially on large datasets or distributed workloads. |
|
|
63
|
+
| **Performance** | Significantly slower, especially on large datasets or distributed workloads. | Native C++ implementation with measured speedups that depend on the workload and hardware. |
|
|
67
64
|
| **Functionality** | Limited to basic COCO format evaluation. | Offers extended metrics, support for new IoU types, compatibility with more datasets (e.g., CrowdPose, LVIS), advanced visualizations, and seamless integration with PyTorch/TorchVision. |
|
|
68
|
-
| **Ease of Use** | Requires manual installation, often with compilation issues.
|
|
69
|
-
| **Visualization** | Basic plotting capabilities.
|
|
65
|
+
| **Ease of Use** | Requires manual installation, often with compilation issues. | Simple `pip install` with pre-built wheels for common CPython/glibc Linux, macOS, and Windows platforms; source builds may be needed elsewhere. |
|
|
66
|
+
| **Visualization** | Basic plotting capabilities. | Advanced error visualization, annotation display, and comprehensive metric analysis tools. |
|
|
70
67
|
|
|
71
|
-
|
|
68
|
+
______________________________________________________________________
|
|
72
69
|
|
|
73
70
|
**Key Benefits of Faster-COCO-Eval:**
|
|
74
71
|
|
|
75
|
-
✅ **Blazing Fast Performance** - Evaluate large datasets in minutes instead of hours
|
|
76
|
-
✅ **Reliable & Trusted** - Extensive test coverage ensures consistent, reproducible results
|
|
77
|
-
✅ **Modern Features** - Support for latest CV tasks, IoU types, and dataset formats
|
|
78
|
-
✅ **Easy to Use** - Drop-in replacement for pycocotools with enhanced API
|
|
79
|
-
✅ **Comprehensive Visualization** - Understand your model's performance with beautiful, informative plots
|
|
72
|
+
✅ **Blazing Fast Performance** - Evaluate large datasets in minutes instead of hours
|
|
73
|
+
✅ **Reliable & Trusted** - Extensive test coverage ensures consistent, reproducible results
|
|
74
|
+
✅ **Modern Features** - Support for latest CV tasks, IoU types, and dataset formats
|
|
75
|
+
✅ **Easy to Use** - Drop-in replacement for pycocotools with enhanced API
|
|
76
|
+
✅ **Comprehensive Visualization** - Understand your model's performance with beautiful, informative plots
|
|
80
77
|
|
|
81
78
|
**Join thousands of computer vision researchers and engineers who have already switched to Faster-COCO-Eval!**
|
|
82
79
|
|
|
@@ -121,6 +118,8 @@ from pycocotools.coco import COCO
|
|
|
121
118
|
from pycocotools.cocoeval import COCOeval
|
|
122
119
|
|
|
123
120
|
# Load annotations and predictions
|
|
121
|
+
anno_json = "path/to/annotations.json"
|
|
122
|
+
pred_json = "path/to/predictions.json"
|
|
124
123
|
anno = COCO(str(anno_json)) # Annotations file
|
|
125
124
|
pred = anno.loadRes(str(pred_json)) # Predictions file
|
|
126
125
|
|
|
@@ -137,22 +136,21 @@ val.accumulate()
|
|
|
137
136
|
val.summarize()
|
|
138
137
|
```
|
|
139
138
|
|
|
140
|
-
**That's it! Your existing code
|
|
139
|
+
**That's it! Your existing code runs through the native evaluator with no API changes.**
|
|
141
140
|
|
|
142
141
|
## ⚡ Blazing Fast Performance
|
|
143
142
|
|
|
144
|
-
Faster-COCO-Eval
|
|
143
|
+
Faster-COCO-Eval uses a native C++ implementation. The measured result below is one local reference point; real-world speedup varies with dataset size, metric, and hardware.
|
|
145
144
|
|
|
146
145
|
### Real-World Performance Benchmark
|
|
147
146
|
|
|
148
|
-
|
|
147
|
+
Local reference measurement: 100 synthetic images, 15 ground-truth boxes and 100 detections per image, five categories, bbox evaluation, seven timed samples after two warmups. The measurement was run on 2026-08-23 on macOS 26.6.1 arm64, Python 3.10.11, and NumPy 2.2.6. The implementations' `eval["precision"]` arrays matched within the comparison tolerance.
|
|
149
148
|
|
|
150
|
-
| Evaluation Type | Faster-COCO-Eval (sec)
|
|
151
|
-
|
|
152
|
-
| Bounding Boxes |
|
|
153
|
-
| Segmentation | 7.413 | 24.434 | **3.3x** |
|
|
149
|
+
| Evaluation Type | Faster-COCO-Eval (sec) | pycocotools (sec) | Speedup |
|
|
150
|
+
| --------------- | ----------------------- | ----------------------- | --------- |
|
|
151
|
+
| Bounding Boxes | 0.053977 (MAD 0.001883) | 0.296960 (MAD 0.001667) | **5.50x** |
|
|
154
152
|
|
|
155
|
-
|
|
153
|
+
This local measurement is hardware- and workload-dependent; it is not a universal performance guarantee.
|
|
156
154
|
|
|
157
155
|
### Colab Examples
|
|
158
156
|
|
|
@@ -166,6 +164,7 @@ See the performance in action:
|
|
|
166
164
|
Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
167
165
|
|
|
168
166
|
### Core Evaluation
|
|
167
|
+
|
|
169
168
|
- **Drop-in pycocotools replacement** - No code changes needed
|
|
170
169
|
- **Support for all COCO metric types**: bbox, segm, keypoints
|
|
171
170
|
- **LVIS (Large Vocabulary Instance Segmentation) evaluation**
|
|
@@ -173,6 +172,7 @@ Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
|
173
172
|
- **Multiple IoU types**: standard, rotated, and custom IoU definitions
|
|
174
173
|
|
|
175
174
|
### Advanced Visualization
|
|
175
|
+
|
|
176
176
|
- **Error visualization**: See where your model is making mistakes
|
|
177
177
|
- **Annotation display**: Visualize ground truth and predictions together
|
|
178
178
|
- **Metric curves**: Precision-recall curves, class-wise performance
|
|
@@ -180,12 +180,14 @@ Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
|
180
180
|
- **Interactive Jupyter notebook examples**
|
|
181
181
|
|
|
182
182
|
### Modern Integrations
|
|
183
|
+
|
|
183
184
|
- **PyTorch/TorchVision compatibility**
|
|
184
|
-
- **Seamless integration with mmdetection
|
|
185
|
+
- **Seamless integration with mmdetection and YOLO frameworks**
|
|
185
186
|
- **Distributed evaluation support**
|
|
186
187
|
- **Memory optimized for large datasets**
|
|
187
188
|
|
|
188
189
|
### Additional Tools
|
|
190
|
+
|
|
189
191
|
- **Boundary evaluation for segmentation tasks**
|
|
190
192
|
- **Custom dataset support**
|
|
191
193
|
- **Comprehensive API documentation**
|
|
@@ -198,13 +200,13 @@ Faster-COCO-Eval prioritizes **correctness and reliability** through extensive t
|
|
|
198
200
|
### Comprehensive Test Suite
|
|
199
201
|
|
|
200
202
|
- **90+ automated tests** covering all functionality
|
|
201
|
-
- **
|
|
202
|
-
- **Continuous integration** on Python 3.
|
|
203
|
+
- **Numerical parity checks** against pycocotools across the supported metrics
|
|
204
|
+
- **Continuous integration** on Python 3.10-3.13
|
|
203
205
|
- **Edge case coverage** including boundary conditions and error handling
|
|
204
206
|
|
|
205
207
|
### Extensive PyCocoTools Comparison
|
|
206
208
|
|
|
207
|
-
New comprehensive tests validate **
|
|
209
|
+
New comprehensive tests validate **numerical parity within a strict tolerance** with pycocotools:
|
|
208
210
|
|
|
209
211
|
- **Object Detection**: Tests with 10-100 images, hundreds to thousands of annotations
|
|
210
212
|
- **Instance Segmentation**: RLE mask encoding and pixel-level IoU validation
|
|
@@ -212,7 +214,7 @@ New comprehensive tests validate **exact numerical equality** with pycocotools:
|
|
|
212
214
|
- **Multiple Scenarios**: Small/medium/large objects, various confidence distributions
|
|
213
215
|
- **Edge Cases**: Perfect predictions, low-confidence detections, mixed object sizes
|
|
214
216
|
|
|
215
|
-
|
|
217
|
+
The comparison suite checks numerical parity between faster_coco_eval and pycocotools, giving you confidence to use this library as a drop-in replacement. See the benchmark above for one measured bbox workload; it is not a universal performance guarantee.
|
|
216
218
|
|
|
217
219
|
See [tests/README.md](tests/README.md) for detailed test documentation.
|
|
218
220
|
|
|
@@ -16,20 +16,20 @@
|
|
|
16
16
|
| ------------------------------ | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
17
17
|
| **Support & Development** | Outdated and not actively maintained. Issues and incompatibilities arise with new releases. | Actively maintained, continuously evolving, and regularly updated with new features and bug fixes. |
|
|
18
18
|
| **Transparency & Reliability** | Lacks comprehensive testing, making updates risky and results less predictable. | Emphasizes extensive test coverage and code quality, ensuring trustworthy and reliable results. |
|
|
19
|
-
| **Performance** | Significantly slower, especially on large datasets or distributed workloads. |
|
|
19
|
+
| **Performance** | Significantly slower, especially on large datasets or distributed workloads. | Native C++ implementation with measured speedups that depend on the workload and hardware. |
|
|
20
20
|
| **Functionality** | Limited to basic COCO format evaluation. | Offers extended metrics, support for new IoU types, compatibility with more datasets (e.g., CrowdPose, LVIS), advanced visualizations, and seamless integration with PyTorch/TorchVision. |
|
|
21
|
-
| **Ease of Use** | Requires manual installation, often with compilation issues.
|
|
22
|
-
| **Visualization** | Basic plotting capabilities.
|
|
21
|
+
| **Ease of Use** | Requires manual installation, often with compilation issues. | Simple `pip install` with pre-built wheels for common CPython/glibc Linux, macOS, and Windows platforms; source builds may be needed elsewhere. |
|
|
22
|
+
| **Visualization** | Basic plotting capabilities. | Advanced error visualization, annotation display, and comprehensive metric analysis tools. |
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
______________________________________________________________________
|
|
25
25
|
|
|
26
26
|
**Key Benefits of Faster-COCO-Eval:**
|
|
27
27
|
|
|
28
|
-
✅ **Blazing Fast Performance** - Evaluate large datasets in minutes instead of hours
|
|
29
|
-
✅ **Reliable & Trusted** - Extensive test coverage ensures consistent, reproducible results
|
|
30
|
-
✅ **Modern Features** - Support for latest CV tasks, IoU types, and dataset formats
|
|
31
|
-
✅ **Easy to Use** - Drop-in replacement for pycocotools with enhanced API
|
|
32
|
-
✅ **Comprehensive Visualization** - Understand your model's performance with beautiful, informative plots
|
|
28
|
+
✅ **Blazing Fast Performance** - Evaluate large datasets in minutes instead of hours
|
|
29
|
+
✅ **Reliable & Trusted** - Extensive test coverage ensures consistent, reproducible results
|
|
30
|
+
✅ **Modern Features** - Support for latest CV tasks, IoU types, and dataset formats
|
|
31
|
+
✅ **Easy to Use** - Drop-in replacement for pycocotools with enhanced API
|
|
32
|
+
✅ **Comprehensive Visualization** - Understand your model's performance with beautiful, informative plots
|
|
33
33
|
|
|
34
34
|
**Join thousands of computer vision researchers and engineers who have already switched to Faster-COCO-Eval!**
|
|
35
35
|
|
|
@@ -74,6 +74,8 @@ from pycocotools.coco import COCO
|
|
|
74
74
|
from pycocotools.cocoeval import COCOeval
|
|
75
75
|
|
|
76
76
|
# Load annotations and predictions
|
|
77
|
+
anno_json = "path/to/annotations.json"
|
|
78
|
+
pred_json = "path/to/predictions.json"
|
|
77
79
|
anno = COCO(str(anno_json)) # Annotations file
|
|
78
80
|
pred = anno.loadRes(str(pred_json)) # Predictions file
|
|
79
81
|
|
|
@@ -90,22 +92,21 @@ val.accumulate()
|
|
|
90
92
|
val.summarize()
|
|
91
93
|
```
|
|
92
94
|
|
|
93
|
-
**That's it! Your existing code
|
|
95
|
+
**That's it! Your existing code runs through the native evaluator with no API changes.**
|
|
94
96
|
|
|
95
97
|
## ⚡ Blazing Fast Performance
|
|
96
98
|
|
|
97
|
-
Faster-COCO-Eval
|
|
99
|
+
Faster-COCO-Eval uses a native C++ implementation. The measured result below is one local reference point; real-world speedup varies with dataset size, metric, and hardware.
|
|
98
100
|
|
|
99
101
|
### Real-World Performance Benchmark
|
|
100
102
|
|
|
101
|
-
|
|
103
|
+
Local reference measurement: 100 synthetic images, 15 ground-truth boxes and 100 detections per image, five categories, bbox evaluation, seven timed samples after two warmups. The measurement was run on 2026-08-23 on macOS 26.6.1 arm64, Python 3.10.11, and NumPy 2.2.6. The implementations' `eval["precision"]` arrays matched within the comparison tolerance.
|
|
102
104
|
|
|
103
|
-
| Evaluation Type | Faster-COCO-Eval (sec)
|
|
104
|
-
|
|
105
|
-
| Bounding Boxes |
|
|
106
|
-
| Segmentation | 7.413 | 24.434 | **3.3x** |
|
|
105
|
+
| Evaluation Type | Faster-COCO-Eval (sec) | pycocotools (sec) | Speedup |
|
|
106
|
+
| --------------- | ----------------------- | ----------------------- | --------- |
|
|
107
|
+
| Bounding Boxes | 0.053977 (MAD 0.001883) | 0.296960 (MAD 0.001667) | **5.50x** |
|
|
107
108
|
|
|
108
|
-
|
|
109
|
+
This local measurement is hardware- and workload-dependent; it is not a universal performance guarantee.
|
|
109
110
|
|
|
110
111
|
### Colab Examples
|
|
111
112
|
|
|
@@ -119,6 +120,7 @@ See the performance in action:
|
|
|
119
120
|
Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
120
121
|
|
|
121
122
|
### Core Evaluation
|
|
123
|
+
|
|
122
124
|
- **Drop-in pycocotools replacement** - No code changes needed
|
|
123
125
|
- **Support for all COCO metric types**: bbox, segm, keypoints
|
|
124
126
|
- **LVIS (Large Vocabulary Instance Segmentation) evaluation**
|
|
@@ -126,6 +128,7 @@ Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
|
126
128
|
- **Multiple IoU types**: standard, rotated, and custom IoU definitions
|
|
127
129
|
|
|
128
130
|
### Advanced Visualization
|
|
131
|
+
|
|
129
132
|
- **Error visualization**: See where your model is making mistakes
|
|
130
133
|
- **Annotation display**: Visualize ground truth and predictions together
|
|
131
134
|
- **Metric curves**: Precision-recall curves, class-wise performance
|
|
@@ -133,12 +136,14 @@ Faster-COCO-Eval goes beyond basic evaluation with these advanced capabilities:
|
|
|
133
136
|
- **Interactive Jupyter notebook examples**
|
|
134
137
|
|
|
135
138
|
### Modern Integrations
|
|
139
|
+
|
|
136
140
|
- **PyTorch/TorchVision compatibility**
|
|
137
|
-
- **Seamless integration with mmdetection
|
|
141
|
+
- **Seamless integration with mmdetection and YOLO frameworks**
|
|
138
142
|
- **Distributed evaluation support**
|
|
139
143
|
- **Memory optimized for large datasets**
|
|
140
144
|
|
|
141
145
|
### Additional Tools
|
|
146
|
+
|
|
142
147
|
- **Boundary evaluation for segmentation tasks**
|
|
143
148
|
- **Custom dataset support**
|
|
144
149
|
- **Comprehensive API documentation**
|
|
@@ -151,13 +156,13 @@ Faster-COCO-Eval prioritizes **correctness and reliability** through extensive t
|
|
|
151
156
|
### Comprehensive Test Suite
|
|
152
157
|
|
|
153
158
|
- **90+ automated tests** covering all functionality
|
|
154
|
-
- **
|
|
155
|
-
- **Continuous integration** on Python 3.
|
|
159
|
+
- **Numerical parity checks** against pycocotools across the supported metrics
|
|
160
|
+
- **Continuous integration** on Python 3.10-3.13
|
|
156
161
|
- **Edge case coverage** including boundary conditions and error handling
|
|
157
162
|
|
|
158
163
|
### Extensive PyCocoTools Comparison
|
|
159
164
|
|
|
160
|
-
New comprehensive tests validate **
|
|
165
|
+
New comprehensive tests validate **numerical parity within a strict tolerance** with pycocotools:
|
|
161
166
|
|
|
162
167
|
- **Object Detection**: Tests with 10-100 images, hundreds to thousands of annotations
|
|
163
168
|
- **Instance Segmentation**: RLE mask encoding and pixel-level IoU validation
|
|
@@ -165,7 +170,7 @@ New comprehensive tests validate **exact numerical equality** with pycocotools:
|
|
|
165
170
|
- **Multiple Scenarios**: Small/medium/large objects, various confidence distributions
|
|
166
171
|
- **Edge Cases**: Perfect predictions, low-confidence detections, mixed object sizes
|
|
167
172
|
|
|
168
|
-
|
|
173
|
+
The comparison suite checks numerical parity between faster_coco_eval and pycocotools, giving you confidence to use this library as a drop-in replacement. See the benchmark above for one measured bbox workload; it is not a universal performance guarantee.
|
|
169
174
|
|
|
170
175
|
See [tests/README.md](tests/README.md) for detailed test documentation.
|
|
171
176
|
|