faster-coco-eval 1.5.1__tar.gz → 1.5.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. {faster_coco_eval-1.5.1/faster_coco_eval.egg-info → faster_coco_eval-1.5.3}/PKG-INFO +11 -14
  2. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/README.md +10 -13
  3. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/coco.py +26 -17
  4. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/cocoeval.py +47 -13
  5. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/faster_eval_api.py +16 -4
  6. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/draw.py +38 -5
  7. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/extra.py +39 -1
  8. faster_coco_eval-1.5.3/faster_coco_eval/extra/utils.py +86 -0
  9. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/version.py +1 -1
  10. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3/faster_coco_eval.egg-info}/PKG-INFO +11 -14
  11. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/SOURCES.txt +2 -0
  12. faster_coco_eval-1.5.3/requirements/extra.txt +1 -0
  13. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/LICENSE +0 -0
  14. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/MANIFEST.in +0 -0
  15. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/coco_eval/cocoeval.cpp +0 -0
  16. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/coco_eval/cocoeval.h +0 -0
  17. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/faster_eval_api/faster_eval_api.cpp +0 -0
  18. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/common/maskApi.c +0 -0
  19. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/common/maskApi.h +0 -0
  20. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/csrc/mask/pycocotools/_mask.pyx +0 -0
  21. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/__init__.py +0 -0
  22. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/__init__.py +0 -0
  23. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/core/mask.py +0 -0
  24. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/__init__.py +0 -0
  25. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/curves.py +0 -0
  26. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval/extra/display.py +0 -0
  27. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/dependency_links.txt +0 -0
  28. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/requires.txt +0 -0
  29. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/faster_coco_eval.egg-info/top_level.txt +0 -0
  30. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/pyproject.toml +0 -0
  31. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/requirements/runtime.txt +0 -0
  32. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/setup.cfg +0 -0
  33. {faster_coco_eval-1.5.1 → faster_coco_eval-1.5.3}/setup.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: faster-coco-eval
3
- Version: 1.5.1
3
+ Version: 1.5.3
4
4
  Summary: Faster interpretation of the original COCOEval
5
5
  Home-page: https://github.com/MiXaiLL76/faster_coco_eval
6
6
  Author: MiXaiLL76
@@ -32,6 +32,9 @@ https://pypi.org/project/faster-coco-eval/)
32
32
  [![docs](https://img.shields.io/badge/docs-latest-blue)](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
33
33
  [![license](https://img.shields.io/github/license/MiXaiLL76/faster_coco_eval.svg)](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
34
34
 
35
+ [![CI - Test](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml/badge.svg)](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
36
+ <!-- [![Coverage](https://codecov.io/github/MiXaiLL76/faster_coco_eval/coverage.svg?branch=main)](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
37
+
35
38
  ## Disclaimer
36
39
 
37
40
  I often use this project, but I saw it abandoned and without a public repository on github.
@@ -49,22 +52,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
49
52
  For our use case with a test dataset of 5000 images from the coco val dataset.
50
53
  Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
51
54
 
52
- Visualization of testing **comparison.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/comparison.ipynb)
53
- Tested with yolo3 model (bbox eval) and yoloact model (segm eval)
55
+ Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
56
+ [colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
57
+ Tested with rtmdet model bbox + segm
54
58
 
55
59
  ### Summary for 5000 imgs
56
60
 
57
- | Type | COCOeval | COCOeval_faster | Profit |
58
- | ---- | ----------- | --------------- | ------------ |
59
- | bbox | 18.477 sec. | 7.345 sec. | ~2.5x faster |
60
- | segm | 29.819 sec. | 15.840 sec. | ~1.7x faster |
61
-
62
- ## Summary for 500 imgs
63
-
64
- | Type | COCOeval | COCOeval_faster | Profit |
65
- | ---- | --------- | --------------- | ------------ |
66
- | bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
67
- | segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
61
+ | Type | faster-coco-eval | pycocotools | Profit |
62
+ | :--- | ---------------: | ----------: | -----: |
63
+ | bbox | 5.812 | 22.72 | 3.909 |
64
+ | segm | 7.413 | 24.434 | 3.296 |
68
65
 
69
66
  ## Feautures
70
67
 
@@ -7,6 +7,9 @@ https://pypi.org/project/faster-coco-eval/)
7
7
  [![docs](https://img.shields.io/badge/docs-latest-blue)](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
8
8
  [![license](https://img.shields.io/github/license/MiXaiLL76/faster_coco_eval.svg)](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
9
9
 
10
+ [![CI - Test](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml/badge.svg)](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
11
+ <!-- [![Coverage](https://codecov.io/github/MiXaiLL76/faster_coco_eval/coverage.svg?branch=main)](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
12
+
10
13
  ## Disclaimer
11
14
 
12
15
  I often use this project, but I saw it abandoned and without a public repository on github.
@@ -24,22 +27,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
24
27
  For our use case with a test dataset of 5000 images from the coco val dataset.
25
28
  Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
26
29
 
27
- Visualization of testing **comparison.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/comparison.ipynb)
28
- Tested with yolo3 model (bbox eval) and yoloact model (segm eval)
30
+ Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
31
+ [colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
32
+ Tested with rtmdet model bbox + segm
29
33
 
30
34
  ### Summary for 5000 imgs
31
35
 
32
- | Type | COCOeval | COCOeval_faster | Profit |
33
- | ---- | ----------- | --------------- | ------------ |
34
- | bbox | 18.477 sec. | 7.345 sec. | ~2.5x faster |
35
- | segm | 29.819 sec. | 15.840 sec. | ~1.7x faster |
36
-
37
- ## Summary for 500 imgs
38
-
39
- | Type | COCOeval | COCOeval_faster | Profit |
40
- | ---- | --------- | --------------- | ------------ |
41
- | bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
42
- | segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
36
+ | Type | faster-coco-eval | pycocotools | Profit |
37
+ | :--- | ---------------: | ----------: | -----: |
38
+ | bbox | 5.812 | 22.72 | 3.909 |
39
+ | segm | 7.413 | 24.434 | 3.296 |
43
40
 
44
41
  ## Feautures
45
42
 
@@ -90,7 +90,7 @@ class COCO:
90
90
  if type(annotation_file) is str:
91
91
  self.dataset = self.load_json(annotation_file)
92
92
  elif type(annotation_file) is dict:
93
- self.dataset = annotation_file
93
+ self.dataset = copy.deepcopy(annotation_file)
94
94
  else:
95
95
  self.dataset = None
96
96
 
@@ -214,21 +214,29 @@ class COCO:
214
214
  cats = self.dataset["categories"]
215
215
  else:
216
216
  cats = self.dataset["categories"]
217
- cats = (
218
- cats
219
- if len(catNms) == 0
220
- else [cat for cat in cats if cat["name"] in catNms]
221
- )
222
- cats = (
223
- cats
224
- if len(supNms) == 0
225
- else [cat for cat in cats if cat["supercategory"] in supNms]
226
- )
227
- cats = (
228
- cats
229
- if len(catIds) == 0
230
- else [cat for cat in cats if cat["id"] in catIds]
231
- )
217
+
218
+ if len(catNms) > 0:
219
+ name_to_cat = {cat.get("name"): cat for cat in cats}
220
+ cats = [
221
+ name_to_cat[label]
222
+ for label in catNms
223
+ if name_to_cat.get(label)
224
+ ]
225
+
226
+ if len(supNms) > 0:
227
+ supercategory_to_cat = {
228
+ cat.get("supercategory"): cat for cat in cats
229
+ }
230
+ cats = [
231
+ supercategory_to_cat[label]
232
+ for label in supNms
233
+ if supercategory_to_cat.get(label)
234
+ ]
235
+
236
+ if len(catIds) > 0:
237
+ id_to_cat = {cat.get("id"): cat for cat in cats}
238
+ cats = [id_to_cat[idx] for idx in catIds if id_to_cat.get(idx)]
239
+
232
240
  ids = [cat["id"] for cat in cats]
233
241
  return ids
234
242
 
@@ -314,7 +322,8 @@ class COCO:
314
322
  elif type(resFile) is np.ndarray:
315
323
  anns = self.loadNumpyAnnotations(resFile)
316
324
  else:
317
- anns = resFile
325
+ anns = copy.deepcopy(resFile)
326
+
318
327
  assert type(anns) is list, "results in not an array of objects"
319
328
 
320
329
  anns = [ann for ann in anns if ann.get("score", 1) >= self.score_tresh]
@@ -70,6 +70,7 @@ class COCOeval:
70
70
  iouType="segm",
71
71
  print_function=logger.debug,
72
72
  extra_calc=False,
73
+ kpt_oks_sigmas=None,
73
74
  ):
74
75
  """Initialize CocoEval using coco APIs for gt and dt :param cocoGt:
75
76
 
@@ -88,7 +89,9 @@ class COCOeval:
88
89
  self.eval: dict = {} # accumulated evaluation results
89
90
  self._gts = defaultdict(list) # gt for evaluation
90
91
  self._dts = defaultdict(list) # dt for evaluation
91
- self.params = Params(iouType=iouType) # parameters
92
+ self.params = Params(
93
+ iouType=iouType, kpt_sigmas=kpt_oks_sigmas
94
+ ) # parameters
92
95
  self._paramsEval: dict = {} # parameters for evaluation
93
96
  self.stats: list = [] # result summarization
94
97
  self.ious: dict = {} # ious between all gts and dts
@@ -100,6 +103,18 @@ class COCOeval:
100
103
  self.params.imgIds = sorted(cocoGt.getImgIds())
101
104
  self.params.catIds = sorted(cocoGt.getCatIds())
102
105
 
106
+ if iouType == "keypoints":
107
+ if cocoDt is not None:
108
+ self.params.catIds = sorted(list(cocoDt.cat_img_map.keys()))
109
+ else:
110
+ self.params.catIds = sorted(
111
+ [
112
+ category_id
113
+ for category_id, category in cocoGt.cats.items()
114
+ if len(category.get("keypoints", []))
115
+ ]
116
+ )
117
+
103
118
  self.print_function = print_function # output print function
104
119
 
105
120
  def _prepare(self):
@@ -136,7 +151,7 @@ class COCOeval:
136
151
  gt["ignore"] = gt["ignore"] if "ignore" in gt else 0
137
152
  gt["ignore"] = "iscrowd" in gt and gt["iscrowd"]
138
153
  if p.iouType == "keypoints":
139
- gt["ignore"] = (gt["num_keypoints"] == 0) or gt["ignore"]
154
+ gt["ignore"] = (gt.get("num_keypoints") == 0) or gt["ignore"]
140
155
  self._gts = defaultdict(list) # gt for evaluation
141
156
  self._dts = defaultdict(list) # dt for evaluation
142
157
  for gt in gts:
@@ -466,16 +481,32 @@ class COCOeval:
466
481
 
467
482
  def _summarizeKps():
468
483
  stats = np.zeros((10,))
469
- stats[0] = _summarize(1, maxDets=20)
470
- stats[1] = _summarize(1, maxDets=20, iouThr=0.5)
471
- stats[2] = _summarize(1, maxDets=20, iouThr=0.75)
472
- stats[3] = _summarize(1, maxDets=20, areaRng="medium")
473
- stats[4] = _summarize(1, maxDets=20, areaRng="large")
474
- stats[5] = _summarize(0, maxDets=20)
475
- stats[6] = _summarize(0, maxDets=20, iouThr=0.5)
476
- stats[7] = _summarize(0, maxDets=20, iouThr=0.75)
477
- stats[8] = _summarize(0, maxDets=20, areaRng="medium")
478
- stats[9] = _summarize(0, maxDets=20, areaRng="large")
484
+ stats[0] = _summarize(1, maxDets=self.params.maxDets[-1])
485
+ stats[1] = _summarize(
486
+ 1, maxDets=self.params.maxDets[-1], iouThr=0.5
487
+ )
488
+ stats[2] = _summarize(
489
+ 1, maxDets=self.params.maxDets[-1], iouThr=0.75
490
+ )
491
+ stats[3] = _summarize(
492
+ 1, maxDets=self.params.maxDets[-1], areaRng="medium"
493
+ )
494
+ stats[4] = _summarize(
495
+ 1, maxDets=self.params.maxDets[-1], areaRng="large"
496
+ )
497
+ stats[5] = _summarize(0, maxDets=self.params.maxDets[-1])
498
+ stats[6] = _summarize(
499
+ 0, maxDets=self.params.maxDets[-1], iouThr=0.5
500
+ )
501
+ stats[7] = _summarize(
502
+ 0, maxDets=self.params.maxDets[-1], iouThr=0.75
503
+ )
504
+ stats[8] = _summarize(
505
+ 0, maxDets=self.params.maxDets[-1], areaRng="medium"
506
+ )
507
+ stats[9] = _summarize(
508
+ 0, maxDets=self.params.maxDets[-1], areaRng="large"
509
+ )
479
510
  return stats
480
511
 
481
512
  if not self.eval:
@@ -534,6 +565,7 @@ class Params:
534
565
  ]
535
566
  self.areaRngLbl = ["all", "medium", "large"]
536
567
  self.useCats = 1
568
+
537
569
  self.kpt_oks_sigmas = (
538
570
  np.array(
539
571
  [
@@ -559,11 +591,13 @@ class Params:
559
591
  / 10.0
560
592
  )
561
593
 
562
- def __init__(self, iouType="segm"):
594
+ def __init__(self, iouType="segm", kpt_sigmas=None):
563
595
  if iouType == "segm" or iouType == "bbox":
564
596
  self.setDetParams()
565
597
  elif iouType == "keypoints":
566
598
  self.setKpParams()
599
+ if kpt_sigmas is not None:
600
+ self.kpt_oks_sigmas = np.array(kpt_sigmas)
567
601
  else:
568
602
  raise Exception("iouType not supported")
569
603
  self.iouType = iouType
@@ -169,7 +169,7 @@ class COCOeval_faster(COCOeval):
169
169
  self.eval["counts"][0], self.eval["counts"][3], -1
170
170
  )
171
171
  )
172
- assert self.ground_truth_matches.shape[1] == len(
172
+ assert self.ground_truth_matches.shape[1] <= len(
173
173
  self.cocoGt.anns
174
174
  )
175
175
 
@@ -178,7 +178,7 @@ class COCOeval_faster(COCOeval):
178
178
  self.eval["counts"][0], self.eval["counts"][3], -1
179
179
  )
180
180
  )
181
- assert self.ground_truth_orig_id.shape[1] == len(
181
+ assert self.ground_truth_orig_id.shape[1] <= len(
182
182
  self.cocoGt.anns
183
183
  )
184
184
  self.math_matches()
@@ -262,9 +262,21 @@ class COCOeval_faster(COCOeval):
262
262
  g.append(gt_ann["bbox"])
263
263
  d.append(dt_ann["bbox"])
264
264
 
265
- return maskUtils.iou(d, g, [0]).max()
265
+ iscrowd = [0 for _ in g]
266
+
267
+ _iou = maskUtils.iou(d, g, iscrowd)
268
+
269
+ if len(_iou) == 0:
270
+ return 0
271
+ else:
272
+ return _iou.max()
266
273
 
267
274
  def compute_mIoU(self, categories=None, raw=False):
275
+ if self.params.iouType == "keypoints":
276
+ return np.array(
277
+ [val.max() for val in self.ious.values() if len(val)]
278
+ ).mean()
279
+
268
280
  g = []
269
281
  d = []
270
282
  s = []
@@ -283,7 +295,7 @@ class COCOeval_faster(COCOeval):
283
295
  else:
284
296
  raise Exception("unknown iouType for iou computation")
285
297
 
286
- iscrowd = [0 for o in g]
298
+ iscrowd = [0 for _ in g]
287
299
 
288
300
  ious = maskUtils.iou(d, g, iscrowd)
289
301
  if raw:
@@ -8,6 +8,7 @@ import plotly.graph_objs as go
8
8
  from PIL import Image
9
9
 
10
10
  from ..core import COCO
11
+ from .utils import convert_ann_rle_to_poly
11
12
 
12
13
  logger = logging.getLogger(__name__)
13
14
 
@@ -25,6 +26,7 @@ def generate_ann_polygon(
25
26
  iouType: str = "bbox",
26
27
  text: Optional[str] = None,
27
28
  legendgroup: Optional[str] = None,
29
+ category_id_to_skeleton: Optional[dict] = None,
28
30
  ):
29
31
  all_x = []
30
32
  all_y = []
@@ -33,13 +35,32 @@ def generate_ann_polygon(
33
35
  x1, y1, w, h = ann["bbox"]
34
36
  all_x = [x1, x1 + w, x1 + w, x1, x1, None]
35
37
  all_y = [y1, y1, y1 + h, y1 + h, y1, None]
36
- else:
38
+ elif iouType == "segm":
39
+ ann = convert_ann_rle_to_poly(ann)
40
+
37
41
  for poly in ann["segmentation"]:
38
42
  if len(poly) > 3:
39
43
  poly += poly[:2]
40
44
  poly = np.array(poly).reshape(-1, 2)
41
45
  all_x += poly[:, 0].tolist() + [None]
42
46
  all_y += poly[:, 1].tolist() + [None]
47
+ elif iouType == "keypoints":
48
+ skeleton = category_id_to_skeleton.get(ann.get("category_id"))
49
+ keypoints = ann.get("keypoints")
50
+
51
+ if (skeleton is None) or (keypoints is None):
52
+ return
53
+
54
+ xyz = np.int0(keypoints).reshape(-1, 3)
55
+ ready_bones = {i: True for i in range(xyz.shape[0])}
56
+ for p1, p2 in skeleton:
57
+ if ready_bones.get(p1 - 1, False) and ready_bones.get(
58
+ p2 - 1, False
59
+ ):
60
+ all_x += [xyz[int(p1 - 1), 0], xyz[int(p2 - 1), 0], None]
61
+ all_y += [xyz[int(p1 - 1), 1], xyz[int(p2 - 1), 1], None]
62
+ else:
63
+ raise ValueError()
43
64
 
44
65
  return go.Scatter(
45
66
  x=all_x,
@@ -100,6 +121,10 @@ def display_image(
100
121
  categories_labels = {
101
122
  category["id"]: category["name"] for _, category in cocoGt.cats.items()
102
123
  }
124
+ category_id_to_skeleton = {
125
+ category["id"]: category.get("skeleton")
126
+ for _, category in cocoGt.cats.items()
127
+ }
103
128
 
104
129
  if len(gt_anns) > 0:
105
130
  for ann in gt_anns.values():
@@ -115,8 +140,10 @@ def display_image(
115
140
  categories_labels[ann["category_id"]],
116
141
  ),
117
142
  legendgroup="fn",
143
+ category_id_to_skeleton=category_id_to_skeleton,
118
144
  )
119
- polygons.append(poly)
145
+ if poly is not None:
146
+ polygons.append(poly)
120
147
  else:
121
148
  if display_gt:
122
149
  poly = generate_ann_polygon(
@@ -128,8 +155,10 @@ def display_image(
128
155
  categories_labels[ann["category_id"]],
129
156
  ),
130
157
  legendgroup="gt",
158
+ category_id_to_skeleton=category_id_to_skeleton,
131
159
  )
132
- polygons.append(poly)
160
+ if poly is not None:
161
+ polygons.append(poly)
133
162
 
134
163
  if len(dt_anns) > 0:
135
164
  for ann in dt_anns.values():
@@ -153,8 +182,10 @@ def display_image(
153
182
  ann["iou"],
154
183
  ),
155
184
  legendgroup="tp",
185
+ category_id_to_skeleton=category_id_to_skeleton,
156
186
  )
157
- polygons.append(poly)
187
+ if poly is not None:
188
+ polygons.append(poly)
158
189
  else:
159
190
  if display_fp:
160
191
  poly = generate_ann_polygon(
@@ -172,8 +203,10 @@ def display_image(
172
203
  ann["score"],
173
204
  ),
174
205
  legendgroup="fp",
206
+ category_id_to_skeleton=category_id_to_skeleton,
175
207
  )
176
- polygons.append(poly)
208
+ if poly is not None:
209
+ polygons.append(poly)
177
210
 
178
211
  fig = px.imshow(
179
212
  im,
@@ -18,6 +18,7 @@ class ExtraEval:
18
18
  iou_tresh: float = 0.0,
19
19
  recall_count: int = 100,
20
20
  useCats: bool = False,
21
+ kpt_oks_sigmas: list = None,
21
22
  ):
22
23
  self.iouType = iouType
23
24
  self.min_score = min_score
@@ -28,16 +29,27 @@ class ExtraEval:
28
29
  self.cocoDt = copy.deepcopy(cocoDt)
29
30
  self.eval = None
30
31
 
32
+ if iouType == "keypoints":
33
+ self.useCats = True
34
+ self.kpt_oks_sigmas = np.array(kpt_oks_sigmas)
35
+ else:
36
+ self.kpt_oks_sigmas = None
37
+
31
38
  assert self.cocoGt is not None, "cocoGt is empty"
32
39
 
33
40
  if (self.cocoGt is not None) and (self.cocoDt is not None):
41
+ self.drop_cocodt_by_score(min_score=min_score)
34
42
  self.evaluate()
35
43
 
36
44
  def evaluate(self):
37
45
  assert self.cocoDt is not None, "cocoDt is empty"
38
46
 
39
47
  cocoEval = COCOeval_faster(
40
- self.cocoGt, self.cocoDt, self.iouType, extra_calc=True
48
+ self.cocoGt,
49
+ self.cocoDt,
50
+ self.iouType,
51
+ extra_calc=True,
52
+ kpt_oks_sigmas=self.kpt_oks_sigmas,
41
53
  )
42
54
  cocoEval.params.maxDets = [len(self.cocoGt.anns)]
43
55
 
@@ -54,3 +66,29 @@ class ExtraEval:
54
66
  cocoEval.accumulate()
55
67
 
56
68
  self.eval = cocoEval.eval
69
+
70
+ def drop_cocodt_by_score(self, min_score: float):
71
+ assert self.cocoDt is not None, "cocoDt is empty"
72
+
73
+ if min_score > 0:
74
+ bad_keys = {}
75
+ bad_images_keys = []
76
+
77
+ for key, ann in self.cocoDt.anns.items():
78
+ if ann["score"] < min_score:
79
+ if bad_keys.get(ann["image_id"]) is None:
80
+ bad_keys[ann["image_id"]] = {}
81
+
82
+ bad_keys[ann["image_id"]][key] = True
83
+
84
+ bad_images_keys.append(ann["image_id"])
85
+
86
+ for image_id in set(bad_images_keys):
87
+ self.cocoDt.imgToAnns[image_id] = [
88
+ ann
89
+ for ann in self.cocoDt.imgToAnns[image_id]
90
+ if bad_keys.get(image_id, {}).get(ann["id"]) is None
91
+ ]
92
+
93
+ for ann_id in bad_keys.get(image_id, {}).keys():
94
+ del self.cocoDt.anns[ann_id]
@@ -0,0 +1,86 @@
1
+ try:
2
+ import cv2
3
+
4
+ opencv_available = True
5
+ except ImportError:
6
+ opencv_available = False
7
+
8
+ import numpy as np
9
+
10
+ import faster_coco_eval.core.mask as mask_util
11
+
12
+
13
+ def conver_mask_to_poly(mask: np.ndarray, bbox: list, boxes_margin=0.1) -> list:
14
+ """Convert mask (uint8) to list of poly as coco style.
15
+
16
+ :param mask (np.ndarray): numpy array in uint8 0-255
17
+ :param bbox (list): x,y,w,h of this ann
18
+ :param boxes_margin (float): margin for increase bbox
19
+ :return: list of poly as coco style
20
+
21
+ """
22
+ x1, y1, w, h = bbox
23
+ x2 = x1 + w
24
+ y2 = y1 + h
25
+
26
+ m_w, m_h = int(w * boxes_margin), int(h * boxes_margin)
27
+ x1 -= m_w
28
+ if x1 < 0:
29
+ x1 = 0
30
+
31
+ y1 -= m_h
32
+ if y1 < 0:
33
+ y1 = 0
34
+
35
+ x2 += m_w
36
+ y2 += m_h
37
+
38
+ coords, _ = cv2.findContours(
39
+ mask[int(y1) : int(y2), int(x1) : int(x2)].astype(np.uint8),
40
+ cv2.RETR_TREE,
41
+ cv2.CHAIN_APPROX_SIMPLE,
42
+ )
43
+
44
+ coords = list(coords)
45
+ if len(coords) > 0:
46
+ for cnt_idx, cnt in enumerate(coords):
47
+ coords[cnt_idx] = cnt + [x1, y1]
48
+
49
+ coords = [
50
+ _cnt.ravel().tolist() for _cnt in coords if _cnt.shape[0] >= 6
51
+ ]
52
+
53
+ return coords
54
+
55
+
56
+ def convert_rle_to_poly(rle: dict, bbox: list):
57
+ """Convert rle (dict) to list of poly as coco style.
58
+
59
+ :param rle (dict): rle of mask image
60
+ :param bbox (list): x,y,w,h of this ann
61
+ :return: list of poly as coco style
62
+
63
+ """
64
+ mask = mask_util.decode(rle) * 255
65
+ return conver_mask_to_poly(mask, bbox)
66
+
67
+
68
+ def convert_ann_rle_to_poly(ann: dict):
69
+ """Convert ann segm from rle to poly style; Save rle in *count* var;
70
+
71
+ :param ann (dict): ann row
72
+ :return: ann
73
+
74
+ """
75
+ if type(ann["segmentation"]) is dict:
76
+ if not opencv_available:
77
+ raise Exception(
78
+ "Your dataset needs to be converted to polygons. Install"
79
+ " **opencv-python** for this."
80
+ )
81
+
82
+ ann["counts"] = ann["segmentation"]
83
+ ann["segmentation"] = convert_rle_to_poly(
84
+ ann["segmentation"], ann["bbox"]
85
+ )
86
+ return ann
@@ -1,2 +1,2 @@
1
- __version__ = "1.5.1"
1
+ __version__ = "1.5.3"
2
2
  __author__ = "MiXaiLL76"
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: faster-coco-eval
3
- Version: 1.5.1
3
+ Version: 1.5.3
4
4
  Summary: Faster interpretation of the original COCOEval
5
5
  Home-page: https://github.com/MiXaiLL76/faster_coco_eval
6
6
  Author: MiXaiLL76
@@ -32,6 +32,9 @@ https://pypi.org/project/faster-coco-eval/)
32
32
  [![docs](https://img.shields.io/badge/docs-latest-blue)](https://github.com/MiXaiLL76/faster_coco_eval/wiki)
33
33
  [![license](https://img.shields.io/github/license/MiXaiLL76/faster_coco_eval.svg)](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/LICENSE)
34
34
 
35
+ [![CI - Test](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml/badge.svg)](https://github.com/MiXaiLL76/faster_coco_eval/actions/workflows/unittest.yml)
36
+ <!-- [![Coverage](https://codecov.io/github/MiXaiLL76/faster_coco_eval/coverage.svg?branch=main)](https://codecov.io/gh/MiXaiLL76/faster_coco_eval) -->
37
+
35
38
  ## Disclaimer
36
39
 
37
40
  I often use this project, but I saw it abandoned and without a public repository on github.
@@ -49,22 +52,16 @@ for coco's AP metrics, especially when dealing with a high number of instances i
49
52
  For our use case with a test dataset of 5000 images from the coco val dataset.
50
53
  Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
51
54
 
52
- Visualization of testing **comparison.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/comparison.ipynb)
53
- Tested with yolo3 model (bbox eval) and yoloact model (segm eval)
55
+ Visualization of testing **colab_example.ipynb** available in directory [examples/comparison](https://github.com/MiXaiLL76/faster_coco_eval/blob/main/examples/comparison/mmdet/colab_example.ipynb)
56
+ [colab_example.ipynb in google collab](https://drive.google.com/file/d/13raC0nRsil797ITJSv3FTRvUvfEK9SD-/view?usp=sharing)
57
+ Tested with rtmdet model bbox + segm
54
58
 
55
59
  ### Summary for 5000 imgs
56
60
 
57
- | Type | COCOeval | COCOeval_faster | Profit |
58
- | ---- | ----------- | --------------- | ------------ |
59
- | bbox | 18.477 sec. | 7.345 sec. | ~2.5x faster |
60
- | segm | 29.819 sec. | 15.840 sec. | ~1.7x faster |
61
-
62
- ## Summary for 500 imgs
63
-
64
- | Type | COCOeval | COCOeval_faster | Profit |
65
- | ---- | --------- | --------------- | ------------ |
66
- | bbox | 3.57 sec. | 2.03 sec. | ~1.7x faster |
67
- | segm | 4.16 sec. | 2.41 sec. | ~1.7x faster |
61
+ | Type | faster-coco-eval | pycocotools | Profit |
62
+ | :--- | ---------------: | ----------: | -----: |
63
+ | bbox | 5.812 | 22.72 | 3.909 |
64
+ | segm | 7.413 | 24.434 | 3.296 |
68
65
 
69
66
  ## Feautures
70
67
 
@@ -26,4 +26,6 @@ faster_coco_eval/extra/curves.py
26
26
  faster_coco_eval/extra/display.py
27
27
  faster_coco_eval/extra/draw.py
28
28
  faster_coco_eval/extra/extra.py
29
+ faster_coco_eval/extra/utils.py
30
+ requirements/extra.txt
29
31
  requirements/runtime.txt
@@ -0,0 +1 @@
1
+ opencv-python-headless