faster-coco-eval 1.5.1__tar.gz → 1.5.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {faster_coco_eval-1.5.1/faster_coco_eval.egg-info → faster_coco_eval-1.5.3}/PKG-INFO +11 -14
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/README.md +10 -13
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/coco.py +26 -17
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/cocoeval.py +47 -13
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/faster_eval_api.py +16 -4
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/draw.py +38 -5
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/extra.py +39 -1
- faster_coco_eval-1.5.3/faster_coco_eval/extra/utils.py +86 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/version.py +1 -1
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3/faster_coco_eval.egg-info}/PKG-INFO +11 -14
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/SOURCES.txt +2 -0
- faster_coco_eval-1.5.3/requirements/extra.txt +1 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/LICENSE +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/MANIFEST.in +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/coco_eval/cocoeval.cpp +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/coco_eval/cocoeval.h +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/faster_eval_api.cpp +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/common/maskApi.c +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/common/maskApi.h +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/pycocotools/_mask.pyx +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/__init__.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/__init__.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/mask.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/__init__.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/curves.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/display.py +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/dependency_links.txt +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/requires.txt +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/top_level.txt +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/pyproject.toml +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/requirements/runtime.txt +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/setup.cfg +0 -0
- {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/setup.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: faster-coco-eval
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.3
|
|
4
4
|
Summary: Faster interpretation of the original COCOEval
|
|
5
5
|
Home-page: https://github.com/MiXaiLL76/faster_coco_eval
|
|
6
6
|
Author: MiXaiLL76
|
|
@@ -32,6 +32,9 @@ https://pypi.org/project/faster-coco-eval/)
|
|
|
32
32
|
[](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
|
|
33
33
|
[](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
|
|
34
34
|
|
|
35
|
+
[](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
|
|
36
|
+
<!-- [](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
|
|
37
|
+
|
|
35
38
|
## Disclaimer
|
|
36
39
|
|
|
37
40
|
I often use this project, but I saw it abandoned and without a public repository on github.
|
|
@@ -49,22 +52,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
|
|
|
49
52
|
For our use case with a test dataset of 5000 images from the coco val dataset.
|
|
50
53
|
Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
|
|
51
54
|
|
|
52
|
-
Visualization of testing **
|
|
53
|
-
|
|
55
|
+
Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
|
|
56
|
+
[colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
|
|
57
|
+
Tested with rtmdet model bbox + segm
|
|
54
58
|
|
|
55
59
|
### Summary for 5000 imgs
|
|
56
60
|
|
|
57
|
-
| Type |
|
|
58
|
-
|
|
|
59
|
-
| bbox |
|
|
60
|
-
| segm |
|
|
61
|
-
|
|
62
|
-
## Summary for 500 imgs
|
|
63
|
-
|
|
64
|
-
| Type | COCOeval | COCOeval_faster | Profit |
|
|
65
|
-
| ---- | --------- | --------------- | ------------ |
|
|
66
|
-
| bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
|
|
67
|
-
| segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
|
|
61
|
+
| Type | faster-coco-eval | pycocotools | Profit |
|
|
62
|
+
| :--- | ---------------: | ----------: | -----: |
|
|
63
|
+
| bbox | 5.812 | 22.72 | 3.909 |
|
|
64
|
+
| segm | 7.413 | 24.434 | 3.296 |
|
|
68
65
|
|
|
69
66
|
## Feautures
|
|
70
67
|
|
|
@@ -7,6 +7,9 @@ https://pypi.org/project/faster-coco-eval/)
|
|
|
7
7
|
[](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
|
|
8
8
|
[](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
|
|
9
9
|
|
|
10
|
+
[](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
|
|
11
|
+
<!-- [](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
|
|
12
|
+
|
|
10
13
|
## Disclaimer
|
|
11
14
|
|
|
12
15
|
I often use this project, but I saw it abandoned and without a public repository on github.
|
|
@@ -24,22 +27,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
|
|
|
24
27
|
For our use case with a test dataset of 5000 images from the coco val dataset.
|
|
25
28
|
Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
|
|
26
29
|
|
|
27
|
-
Visualization of testing **
|
|
28
|
-
|
|
30
|
+
Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
|
|
31
|
+
[colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
|
|
32
|
+
Tested with rtmdet model bbox + segm
|
|
29
33
|
|
|
30
34
|
### Summary for 5000 imgs
|
|
31
35
|
|
|
32
|
-
| Type |
|
|
33
|
-
|
|
|
34
|
-
| bbox |
|
|
35
|
-
| segm |
|
|
36
|
-
|
|
37
|
-
## Summary for 500 imgs
|
|
38
|
-
|
|
39
|
-
| Type | COCOeval | COCOeval_faster | Profit |
|
|
40
|
-
| ---- | --------- | --------------- | ------------ |
|
|
41
|
-
| bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
|
|
42
|
-
| segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
|
|
36
|
+
| Type | faster-coco-eval | pycocotools | Profit |
|
|
37
|
+
| :--- | ---------------: | ----------: | -----: |
|
|
38
|
+
| bbox | 5.812 | 22.72 | 3.909 |
|
|
39
|
+
| segm | 7.413 | 24.434 | 3.296 |
|
|
43
40
|
|
|
44
41
|
## Feautures
|
|
45
42
|
|
|
@@ -90,7 +90,7 @@ class COCO:
|
|
|
90
90
|
if type(annotation_file) is str:
|
|
91
91
|
self.dataset = self.load_json(annotation_file)
|
|
92
92
|
elif type(annotation_file) is dict:
|
|
93
|
-
self.dataset = annotation_file
|
|
93
|
+
self.dataset = copy.deepcopy(annotation_file)
|
|
94
94
|
else:
|
|
95
95
|
self.dataset = None
|
|
96
96
|
|
|
@@ -214,21 +214,29 @@ class COCO:
|
|
|
214
214
|
cats = self.dataset["categories"]
|
|
215
215
|
else:
|
|
216
216
|
cats = self.dataset["categories"]
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
)
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
217
|
+
|
|
218
|
+
if len(catNms) > 0:
|
|
219
|
+
name_to_cat = {cat.get("name"): cat for cat in cats}
|
|
220
|
+
cats = [
|
|
221
|
+
name_to_cat[label]
|
|
222
|
+
for label in catNms
|
|
223
|
+
if name_to_cat.get(label)
|
|
224
|
+
]
|
|
225
|
+
|
|
226
|
+
if len(supNms) > 0:
|
|
227
|
+
supercategory_to_cat = {
|
|
228
|
+
cat.get("supercategory"): cat for cat in cats
|
|
229
|
+
}
|
|
230
|
+
cats = [
|
|
231
|
+
supercategory_to_cat[label]
|
|
232
|
+
for label in supNms
|
|
233
|
+
if supercategory_to_cat.get(label)
|
|
234
|
+
]
|
|
235
|
+
|
|
236
|
+
if len(catIds) > 0:
|
|
237
|
+
id_to_cat = {cat.get("id"): cat for cat in cats}
|
|
238
|
+
cats = [id_to_cat[idx] for idx in catIds if id_to_cat.get(idx)]
|
|
239
|
+
|
|
232
240
|
ids = [cat["id"] for cat in cats]
|
|
233
241
|
return ids
|
|
234
242
|
|
|
@@ -314,7 +322,8 @@ class COCO:
|
|
|
314
322
|
elif type(resFile) is np.ndarray:
|
|
315
323
|
anns = self.loadNumpyAnnotations(resFile)
|
|
316
324
|
else:
|
|
317
|
-
anns = resFile
|
|
325
|
+
anns = copy.deepcopy(resFile)
|
|
326
|
+
|
|
318
327
|
assert type(anns) is list, "results in not an array of objects"
|
|
319
328
|
|
|
320
329
|
anns = [ann for ann in anns if ann.get("score", 1) >= self.score_tresh]
|
|
@@ -70,6 +70,7 @@ class COCOeval:
|
|
|
70
70
|
iouType="segm",
|
|
71
71
|
print_function=logger.debug,
|
|
72
72
|
extra_calc=False,
|
|
73
|
+
kpt_oks_sigmas=None,
|
|
73
74
|
):
|
|
74
75
|
"""Initialize CocoEval using coco APIs for gt and dt :param cocoGt:
|
|
75
76
|
|
|
@@ -88,7 +89,9 @@ class COCOeval:
|
|
|
88
89
|
self.eval: dict = {} # accumulated evaluation results
|
|
89
90
|
self._gts = defaultdict(list) # gt for evaluation
|
|
90
91
|
self._dts = defaultdict(list) # dt for evaluation
|
|
91
|
-
self.params = Params(
|
|
92
|
+
self.params = Params(
|
|
93
|
+
iouType=iouType, kpt_sigmas=kpt_oks_sigmas
|
|
94
|
+
) # parameters
|
|
92
95
|
self._paramsEval: dict = {} # parameters for evaluation
|
|
93
96
|
self.stats: list = [] # result summarization
|
|
94
97
|
self.ious: dict = {} # ious between all gts and dts
|
|
@@ -100,6 +103,18 @@ class COCOeval:
|
|
|
100
103
|
self.params.imgIds = sorted(cocoGt.getImgIds())
|
|
101
104
|
self.params.catIds = sorted(cocoGt.getCatIds())
|
|
102
105
|
|
|
106
|
+
if iouType == "keypoints":
|
|
107
|
+
if cocoDt is not None:
|
|
108
|
+
self.params.catIds = sorted(list(cocoDt.cat_img_map.keys()))
|
|
109
|
+
else:
|
|
110
|
+
self.params.catIds = sorted(
|
|
111
|
+
[
|
|
112
|
+
category_id
|
|
113
|
+
for category_id, category in cocoGt.cats.items()
|
|
114
|
+
if len(category.get("keypoints", []))
|
|
115
|
+
]
|
|
116
|
+
)
|
|
117
|
+
|
|
103
118
|
self.print_function = print_function # output print function
|
|
104
119
|
|
|
105
120
|
def _prepare(self):
|
|
@@ -136,7 +151,7 @@ class COCOeval:
|
|
|
136
151
|
gt["ignore"] = gt["ignore"] if "ignore" in gt else 0
|
|
137
152
|
gt["ignore"] = "iscrowd" in gt and gt["iscrowd"]
|
|
138
153
|
if p.iouType == "keypoints":
|
|
139
|
-
gt["ignore"] = (gt
|
|
154
|
+
gt["ignore"] = (gt.get("num_keypoints") == 0) or gt["ignore"]
|
|
140
155
|
self._gts = defaultdict(list) # gt for evaluation
|
|
141
156
|
self._dts = defaultdict(list) # dt for evaluation
|
|
142
157
|
for gt in gts:
|
|
@@ -466,16 +481,32 @@ class COCOeval:
|
|
|
466
481
|
|
|
467
482
|
def _summarizeKps():
|
|
468
483
|
stats = np.zeros((10,))
|
|
469
|
-
stats[0] = _summarize(1, maxDets=
|
|
470
|
-
stats[1] = _summarize(
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
stats[
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
stats[
|
|
477
|
-
|
|
478
|
-
|
|
484
|
+
stats[0] = _summarize(1, maxDets=self.params.maxDets[-1])
|
|
485
|
+
stats[1] = _summarize(
|
|
486
|
+
1, maxDets=self.params.maxDets[-1], iouThr=0.5
|
|
487
|
+
)
|
|
488
|
+
stats[2] = _summarize(
|
|
489
|
+
1, maxDets=self.params.maxDets[-1], iouThr=0.75
|
|
490
|
+
)
|
|
491
|
+
stats[3] = _summarize(
|
|
492
|
+
1, maxDets=self.params.maxDets[-1], areaRng="medium"
|
|
493
|
+
)
|
|
494
|
+
stats[4] = _summarize(
|
|
495
|
+
1, maxDets=self.params.maxDets[-1], areaRng="large"
|
|
496
|
+
)
|
|
497
|
+
stats[5] = _summarize(0, maxDets=self.params.maxDets[-1])
|
|
498
|
+
stats[6] = _summarize(
|
|
499
|
+
0, maxDets=self.params.maxDets[-1], iouThr=0.5
|
|
500
|
+
)
|
|
501
|
+
stats[7] = _summarize(
|
|
502
|
+
0, maxDets=self.params.maxDets[-1], iouThr=0.75
|
|
503
|
+
)
|
|
504
|
+
stats[8] = _summarize(
|
|
505
|
+
0, maxDets=self.params.maxDets[-1], areaRng="medium"
|
|
506
|
+
)
|
|
507
|
+
stats[9] = _summarize(
|
|
508
|
+
0, maxDets=self.params.maxDets[-1], areaRng="large"
|
|
509
|
+
)
|
|
479
510
|
return stats
|
|
480
511
|
|
|
481
512
|
if not self.eval:
|
|
@@ -534,6 +565,7 @@ class Params:
|
|
|
534
565
|
]
|
|
535
566
|
self.areaRngLbl = ["all", "medium", "large"]
|
|
536
567
|
self.useCats = 1
|
|
568
|
+
|
|
537
569
|
self.kpt_oks_sigmas = (
|
|
538
570
|
np.array(
|
|
539
571
|
[
|
|
@@ -559,11 +591,13 @@ class Params:
|
|
|
559
591
|
/ 10.0
|
|
560
592
|
)
|
|
561
593
|
|
|
562
|
-
def __init__(self, iouType="segm"):
|
|
594
|
+
def __init__(self, iouType="segm", kpt_sigmas=None):
|
|
563
595
|
if iouType == "segm" or iouType == "bbox":
|
|
564
596
|
self.setDetParams()
|
|
565
597
|
elif iouType == "keypoints":
|
|
566
598
|
self.setKpParams()
|
|
599
|
+
if kpt_sigmas is not None:
|
|
600
|
+
self.kpt_oks_sigmas = np.array(kpt_sigmas)
|
|
567
601
|
else:
|
|
568
602
|
raise Exception("iouType not supported")
|
|
569
603
|
self.iouType = iouType
|
|
@@ -169,7 +169,7 @@ class COCOeval_faster(COCOeval):
|
|
|
169
169
|
self.eval["counts"][0], self.eval["counts"][3], -1
|
|
170
170
|
)
|
|
171
171
|
)
|
|
172
|
-
assert self.ground_truth_matches.shape[1]
|
|
172
|
+
assert self.ground_truth_matches.shape[1] <= len(
|
|
173
173
|
self.cocoGt.anns
|
|
174
174
|
)
|
|
175
175
|
|
|
@@ -178,7 +178,7 @@ class COCOeval_faster(COCOeval):
|
|
|
178
178
|
self.eval["counts"][0], self.eval["counts"][3], -1
|
|
179
179
|
)
|
|
180
180
|
)
|
|
181
|
-
assert self.ground_truth_orig_id.shape[1]
|
|
181
|
+
assert self.ground_truth_orig_id.shape[1] <= len(
|
|
182
182
|
self.cocoGt.anns
|
|
183
183
|
)
|
|
184
184
|
self.math_matches()
|
|
@@ -262,9 +262,21 @@ class COCOeval_faster(COCOeval):
|
|
|
262
262
|
g.append(gt_ann["bbox"])
|
|
263
263
|
d.append(dt_ann["bbox"])
|
|
264
264
|
|
|
265
|
-
|
|
265
|
+
iscrowd = [0 for _ in g]
|
|
266
|
+
|
|
267
|
+
_iou = maskUtils.iou(d, g, iscrowd)
|
|
268
|
+
|
|
269
|
+
if len(_iou) == 0:
|
|
270
|
+
return 0
|
|
271
|
+
else:
|
|
272
|
+
return _iou.max()
|
|
266
273
|
|
|
267
274
|
def compute_mIoU(self, categories=None, raw=False):
|
|
275
|
+
if self.params.iouType == "keypoints":
|
|
276
|
+
return np.array(
|
|
277
|
+
[val.max() for val in self.ious.values() if len(val)]
|
|
278
|
+
).mean()
|
|
279
|
+
|
|
268
280
|
g = []
|
|
269
281
|
d = []
|
|
270
282
|
s = []
|
|
@@ -283,7 +295,7 @@ class COCOeval_faster(COCOeval):
|
|
|
283
295
|
else:
|
|
284
296
|
raise Exception("unknown iouType for iou computation")
|
|
285
297
|
|
|
286
|
-
iscrowd = [0 for
|
|
298
|
+
iscrowd = [0 for _ in g]
|
|
287
299
|
|
|
288
300
|
ious = maskUtils.iou(d, g, iscrowd)
|
|
289
301
|
if raw:
|
|
@@ -8,6 +8,7 @@ import plotly.graph_objs as go
|
|
|
8
8
|
from PIL import Image
|
|
9
9
|
|
|
10
10
|
from ..core import COCO
|
|
11
|
+
from .utils import convert_ann_rle_to_poly
|
|
11
12
|
|
|
12
13
|
logger = logging.getLogger(__name__)
|
|
13
14
|
|
|
@@ -25,6 +26,7 @@ def generate_ann_polygon(
|
|
|
25
26
|
iouType: str = "bbox",
|
|
26
27
|
text: Optional[str] = None,
|
|
27
28
|
legendgroup: Optional[str] = None,
|
|
29
|
+
category_id_to_skeleton: Optional[dict] = None,
|
|
28
30
|
):
|
|
29
31
|
all_x = []
|
|
30
32
|
all_y = []
|
|
@@ -33,13 +35,32 @@ def generate_ann_polygon(
|
|
|
33
35
|
x1, y1, w, h = ann["bbox"]
|
|
34
36
|
all_x = [x1, x1 + w, x1 + w, x1, x1, None]
|
|
35
37
|
all_y = [y1, y1, y1 + h, y1 + h, y1, None]
|
|
36
|
-
|
|
38
|
+
elif iouType == "segm":
|
|
39
|
+
ann = convert_ann_rle_to_poly(ann)
|
|
40
|
+
|
|
37
41
|
for poly in ann["segmentation"]:
|
|
38
42
|
if len(poly) > 3:
|
|
39
43
|
poly += poly[:2]
|
|
40
44
|
poly = np.array(poly).reshape(-1, 2)
|
|
41
45
|
all_x += poly[:, 0].tolist() + [None]
|
|
42
46
|
all_y += poly[:, 1].tolist() + [None]
|
|
47
|
+
elif iouType == "keypoints":
|
|
48
|
+
skeleton = category_id_to_skeleton.get(ann.get("category_id"))
|
|
49
|
+
keypoints = ann.get("keypoints")
|
|
50
|
+
|
|
51
|
+
if (skeleton is None) or (keypoints is None):
|
|
52
|
+
return
|
|
53
|
+
|
|
54
|
+
xyz = np.int0(keypoints).reshape(-1, 3)
|
|
55
|
+
ready_bones = {i: True for i in range(xyz.shape[0])}
|
|
56
|
+
for p1, p2 in skeleton:
|
|
57
|
+
if ready_bones.get(p1 - 1, False) and ready_bones.get(
|
|
58
|
+
p2 - 1, False
|
|
59
|
+
):
|
|
60
|
+
all_x += [xyz[int(p1 - 1), 0], xyz[int(p2 - 1), 0], None]
|
|
61
|
+
all_y += [xyz[int(p1 - 1), 1], xyz[int(p2 - 1), 1], None]
|
|
62
|
+
else:
|
|
63
|
+
raise ValueError()
|
|
43
64
|
|
|
44
65
|
return go.Scatter(
|
|
45
66
|
x=all_x,
|
|
@@ -100,6 +121,10 @@ def display_image(
|
|
|
100
121
|
categories_labels = {
|
|
101
122
|
category["id"]: category["name"] for _, category in cocoGt.cats.items()
|
|
102
123
|
}
|
|
124
|
+
category_id_to_skeleton = {
|
|
125
|
+
category["id"]: category.get("skeleton")
|
|
126
|
+
for _, category in cocoGt.cats.items()
|
|
127
|
+
}
|
|
103
128
|
|
|
104
129
|
if len(gt_anns) > 0:
|
|
105
130
|
for ann in gt_anns.values():
|
|
@@ -115,8 +140,10 @@ def display_image(
|
|
|
115
140
|
categories_labels[ann["category_id"]],
|
|
116
141
|
),
|
|
117
142
|
legendgroup="fn",
|
|
143
|
+
category_id_to_skeleton=category_id_to_skeleton,
|
|
118
144
|
)
|
|
119
|
-
|
|
145
|
+
if poly is not None:
|
|
146
|
+
polygons.append(poly)
|
|
120
147
|
else:
|
|
121
148
|
if display_gt:
|
|
122
149
|
poly = generate_ann_polygon(
|
|
@@ -128,8 +155,10 @@ def display_image(
|
|
|
128
155
|
categories_labels[ann["category_id"]],
|
|
129
156
|
),
|
|
130
157
|
legendgroup="gt",
|
|
158
|
+
category_id_to_skeleton=category_id_to_skeleton,
|
|
131
159
|
)
|
|
132
|
-
|
|
160
|
+
if poly is not None:
|
|
161
|
+
polygons.append(poly)
|
|
133
162
|
|
|
134
163
|
if len(dt_anns) > 0:
|
|
135
164
|
for ann in dt_anns.values():
|
|
@@ -153,8 +182,10 @@ def display_image(
|
|
|
153
182
|
ann["iou"],
|
|
154
183
|
),
|
|
155
184
|
legendgroup="tp",
|
|
185
|
+
category_id_to_skeleton=category_id_to_skeleton,
|
|
156
186
|
)
|
|
157
|
-
|
|
187
|
+
if poly is not None:
|
|
188
|
+
polygons.append(poly)
|
|
158
189
|
else:
|
|
159
190
|
if display_fp:
|
|
160
191
|
poly = generate_ann_polygon(
|
|
@@ -172,8 +203,10 @@ def display_image(
|
|
|
172
203
|
ann["score"],
|
|
173
204
|
),
|
|
174
205
|
legendgroup="fp",
|
|
206
|
+
category_id_to_skeleton=category_id_to_skeleton,
|
|
175
207
|
)
|
|
176
|
-
|
|
208
|
+
if poly is not None:
|
|
209
|
+
polygons.append(poly)
|
|
177
210
|
|
|
178
211
|
fig = px.imshow(
|
|
179
212
|
im,
|
|
@@ -18,6 +18,7 @@ class ExtraEval:
|
|
|
18
18
|
iou_tresh: float = 0.0,
|
|
19
19
|
recall_count: int = 100,
|
|
20
20
|
useCats: bool = False,
|
|
21
|
+
kpt_oks_sigmas: list = None,
|
|
21
22
|
):
|
|
22
23
|
self.iouType = iouType
|
|
23
24
|
self.min_score = min_score
|
|
@@ -28,16 +29,27 @@ class ExtraEval:
|
|
|
28
29
|
self.cocoDt = copy.deepcopy(cocoDt)
|
|
29
30
|
self.eval = None
|
|
30
31
|
|
|
32
|
+
if iouType == "keypoints":
|
|
33
|
+
self.useCats = True
|
|
34
|
+
self.kpt_oks_sigmas = np.array(kpt_oks_sigmas)
|
|
35
|
+
else:
|
|
36
|
+
self.kpt_oks_sigmas = None
|
|
37
|
+
|
|
31
38
|
assert self.cocoGt is not None, "cocoGt is empty"
|
|
32
39
|
|
|
33
40
|
if (self.cocoGt is not None) and (self.cocoDt is not None):
|
|
41
|
+
self.drop_cocodt_by_score(min_score=min_score)
|
|
34
42
|
self.evaluate()
|
|
35
43
|
|
|
36
44
|
def evaluate(self):
|
|
37
45
|
assert self.cocoDt is not None, "cocoDt is empty"
|
|
38
46
|
|
|
39
47
|
cocoEval = COCOeval_faster(
|
|
40
|
-
self.cocoGt,
|
|
48
|
+
self.cocoGt,
|
|
49
|
+
self.cocoDt,
|
|
50
|
+
self.iouType,
|
|
51
|
+
extra_calc=True,
|
|
52
|
+
kpt_oks_sigmas=self.kpt_oks_sigmas,
|
|
41
53
|
)
|
|
42
54
|
cocoEval.params.maxDets = [len(self.cocoGt.anns)]
|
|
43
55
|
|
|
@@ -54,3 +66,29 @@ class ExtraEval:
|
|
|
54
66
|
cocoEval.accumulate()
|
|
55
67
|
|
|
56
68
|
self.eval = cocoEval.eval
|
|
69
|
+
|
|
70
|
+
def drop_cocodt_by_score(self, min_score: float):
|
|
71
|
+
assert self.cocoDt is not None, "cocoDt is empty"
|
|
72
|
+
|
|
73
|
+
if min_score > 0:
|
|
74
|
+
bad_keys = {}
|
|
75
|
+
bad_images_keys = []
|
|
76
|
+
|
|
77
|
+
for key, ann in self.cocoDt.anns.items():
|
|
78
|
+
if ann["score"] < min_score:
|
|
79
|
+
if bad_keys.get(ann["image_id"]) is None:
|
|
80
|
+
bad_keys[ann["image_id"]] = {}
|
|
81
|
+
|
|
82
|
+
bad_keys[ann["image_id"]][key] = True
|
|
83
|
+
|
|
84
|
+
bad_images_keys.append(ann["image_id"])
|
|
85
|
+
|
|
86
|
+
for image_id in set(bad_images_keys):
|
|
87
|
+
self.cocoDt.imgToAnns[image_id] = [
|
|
88
|
+
ann
|
|
89
|
+
for ann in self.cocoDt.imgToAnns[image_id]
|
|
90
|
+
if bad_keys.get(image_id, {}).get(ann["id"]) is None
|
|
91
|
+
]
|
|
92
|
+
|
|
93
|
+
for ann_id in bad_keys.get(image_id, {}).keys():
|
|
94
|
+
del self.cocoDt.anns[ann_id]
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
try:
|
|
2
|
+
import cv2
|
|
3
|
+
|
|
4
|
+
opencv_available = True
|
|
5
|
+
except ImportError:
|
|
6
|
+
opencv_available = False
|
|
7
|
+
|
|
8
|
+
import numpy as np
|
|
9
|
+
|
|
10
|
+
import faster_coco_eval.core.mask as mask_util
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
def conver_mask_to_poly(mask: np.ndarray, bbox: list, boxes_margin=0.1) -> list:
|
|
14
|
+
"""Convert mask (uint8) to list of poly as coco style.
|
|
15
|
+
|
|
16
|
+
:param mask (np.ndarray): numpy array in uint8 0-255
|
|
17
|
+
:param bbox (list): x,y,w,h of this ann
|
|
18
|
+
:param boxes_margin (float): margin for increase bbox
|
|
19
|
+
:return: list of poly as coco style
|
|
20
|
+
|
|
21
|
+
"""
|
|
22
|
+
x1, y1, w, h = bbox
|
|
23
|
+
x2 = x1 + w
|
|
24
|
+
y2 = y1 + h
|
|
25
|
+
|
|
26
|
+
m_w, m_h = int(w * boxes_margin), int(h * boxes_margin)
|
|
27
|
+
x1 -= m_w
|
|
28
|
+
if x1 < 0:
|
|
29
|
+
x1 = 0
|
|
30
|
+
|
|
31
|
+
y1 -= m_h
|
|
32
|
+
if y1 < 0:
|
|
33
|
+
y1 = 0
|
|
34
|
+
|
|
35
|
+
x2 += m_w
|
|
36
|
+
y2 += m_h
|
|
37
|
+
|
|
38
|
+
coords, _ = cv2.findContours(
|
|
39
|
+
mask[int(y1) : int(y2), int(x1) : int(x2)].astype(np.uint8),
|
|
40
|
+
cv2.RETR_TREE,
|
|
41
|
+
cv2.CHAIN_APPROX_SIMPLE,
|
|
42
|
+
)
|
|
43
|
+
|
|
44
|
+
coords = list(coords)
|
|
45
|
+
if len(coords) > 0:
|
|
46
|
+
for cnt_idx, cnt in enumerate(coords):
|
|
47
|
+
coords[cnt_idx] = cnt + [x1, y1]
|
|
48
|
+
|
|
49
|
+
coords = [
|
|
50
|
+
_cnt.ravel().tolist() for _cnt in coords if _cnt.shape[0] >= 6
|
|
51
|
+
]
|
|
52
|
+
|
|
53
|
+
return coords
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def convert_rle_to_poly(rle: dict, bbox: list):
|
|
57
|
+
"""Convert rle (dict) to list of poly as coco style.
|
|
58
|
+
|
|
59
|
+
:param rle (dict): rle of mask image
|
|
60
|
+
:param bbox (list): x,y,w,h of this ann
|
|
61
|
+
:return: list of poly as coco style
|
|
62
|
+
|
|
63
|
+
"""
|
|
64
|
+
mask = mask_util.decode(rle) * 255
|
|
65
|
+
return conver_mask_to_poly(mask, bbox)
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
def convert_ann_rle_to_poly(ann: dict):
|
|
69
|
+
"""Convert ann segm from rle to poly style; Save rle in *count* var;
|
|
70
|
+
|
|
71
|
+
:param ann (dict): ann row
|
|
72
|
+
:return: ann
|
|
73
|
+
|
|
74
|
+
"""
|
|
75
|
+
if type(ann["segmentation"]) is dict:
|
|
76
|
+
if not opencv_available:
|
|
77
|
+
raise Exception(
|
|
78
|
+
"Your dataset needs to be converted to polygons. Install"
|
|
79
|
+
" **opencv-python** for this."
|
|
80
|
+
)
|
|
81
|
+
|
|
82
|
+
ann["counts"] = ann["segmentation"]
|
|
83
|
+
ann["segmentation"] = convert_rle_to_poly(
|
|
84
|
+
ann["segmentation"], ann["bbox"]
|
|
85
|
+
)
|
|
86
|
+
return ann
|
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
__version__ = "1.5.
|
|
1
|
+
__version__ = "1.5.3"
|
|
2
2
|
__author__ = "MiXaiLL76"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: faster-coco-eval
|
|
3
|
-
Version: 1.5.
|
|
3
|
+
Version: 1.5.3
|
|
4
4
|
Summary: Faster interpretation of the original COCOEval
|
|
5
5
|
Home-page: https://github.com/MiXaiLL76/faster_coco_eval
|
|
6
6
|
Author: MiXaiLL76
|
|
@@ -32,6 +32,9 @@ https://pypi.org/project/faster-coco-eval/)
|
|
|
32
32
|
[](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
|
|
33
33
|
[](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
|
|
34
34
|
|
|
35
|
+
[](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
|
|
36
|
+
<!-- [](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
|
|
37
|
+
|
|
35
38
|
## Disclaimer
|
|
36
39
|
|
|
37
40
|
I often use this project, but I saw it abandoned and without a public repository on github.
|
|
@@ -49,22 +52,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
|
|
|
49
52
|
For our use case with a test dataset of 5000 images from the coco val dataset.
|
|
50
53
|
Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
|
|
51
54
|
|
|
52
|
-
Visualization of testing **
|
|
53
|
-
|
|
55
|
+
Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
|
|
56
|
+
[colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
|
|
57
|
+
Tested with rtmdet model bbox + segm
|
|
54
58
|
|
|
55
59
|
### Summary for 5000 imgs
|
|
56
60
|
|
|
57
|
-
| Type |
|
|
58
|
-
|
|
|
59
|
-
| bbox |
|
|
60
|
-
| segm |
|
|
61
|
-
|
|
62
|
-
## Summary for 500 imgs
|
|
63
|
-
|
|
64
|
-
| Type | COCOeval | COCOeval_faster | Profit |
|
|
65
|
-
| ---- | --------- | --------------- | ------------ |
|
|
66
|
-
| bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
|
|
67
|
-
| segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
|
|
61
|
+
| Type | faster-coco-eval | pycocotools | Profit |
|
|
62
|
+
| :--- | ---------------: | ----------: | -----: |
|
|
63
|
+
| bbox | 5.812 | 22.72 | 3.909 |
|
|
64
|
+
| segm | 7.413 | 24.434 | 3.296 |
|
|
68
65
|
|
|
69
66
|
## Feautures
|
|
70
67
|
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
opencv-python-headless
|
|
File without changes
|
|
File without changes
|
{faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/coco_eval/cocoeval.cpp
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/dependency_links.txt
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|