frbench 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- frbench-1.0.0/LICENSE +21 -0
- frbench-1.0.0/PKG-INFO +347 -0
- frbench-1.0.0/README.md +318 -0
- frbench-1.0.0/frbench/__init__.py +58 -0
- frbench-1.0.0/frbench/_config.py +51 -0
- frbench-1.0.0/frbench/_exceptions.py +17 -0
- frbench-1.0.0/frbench/backbones/__init__.py +154 -0
- frbench-1.0.0/frbench/backbones/convnext.py +777 -0
- frbench-1.0.0/frbench/backbones/densenet.py +399 -0
- frbench-1.0.0/frbench/backbones/efficientnet.py +805 -0
- frbench-1.0.0/frbench/backbones/ir.py +429 -0
- frbench-1.0.0/frbench/backbones/irse.py +424 -0
- frbench-1.0.0/frbench/backbones/mobilefacenet.py +575 -0
- frbench-1.0.0/frbench/backbones/mobilenet.py +2435 -0
- frbench-1.0.0/frbench/backbones/mobilevit_commons.py +538 -0
- frbench-1.0.0/frbench/backbones/mobilevit_v1.py +262 -0
- frbench-1.0.0/frbench/backbones/mobilevit_v2.py +293 -0
- frbench-1.0.0/frbench/backbones/mobilevit_v3.py +528 -0
- frbench-1.0.0/frbench/backbones/resnet.py +424 -0
- frbench-1.0.0/frbench/backbones/resnetv2.py +493 -0
- frbench-1.0.0/frbench/backbones/swin_commons.py +691 -0
- frbench-1.0.0/frbench/backbones/swin_mlp.py +629 -0
- frbench-1.0.0/frbench/backbones/swin_v1.py +383 -0
- frbench-1.0.0/frbench/backbones/swin_v2.py +925 -0
- frbench-1.0.0/frbench/backbones/utils.py +231 -0
- frbench-1.0.0/frbench/cli.py +76 -0
- frbench-1.0.0/frbench/fr.py +305 -0
- frbench-1.0.0/frbench/types.py +43 -0
- frbench-1.0.0/frbench/utils/__init__.py +0 -0
- frbench-1.0.0/frbench/utils/download.py +303 -0
- frbench-1.0.0/frbench/utils/log.py +103 -0
- frbench-1.0.0/frbench/utils/postprocess.py +56 -0
- frbench-1.0.0/frbench/utils/preprocess.py +442 -0
- frbench-1.0.0/frbench/utils/retinaface.py +390 -0
- frbench-1.0.0/frbench/utils/update_check.py +147 -0
- frbench-1.0.0/frbench.egg-info/PKG-INFO +347 -0
- frbench-1.0.0/frbench.egg-info/SOURCES.txt +41 -0
- frbench-1.0.0/frbench.egg-info/dependency_links.txt +1 -0
- frbench-1.0.0/frbench.egg-info/entry_points.txt +2 -0
- frbench-1.0.0/frbench.egg-info/requires.txt +9 -0
- frbench-1.0.0/frbench.egg-info/top_level.txt +1 -0
- frbench-1.0.0/pyproject.toml +49 -0
- frbench-1.0.0/setup.cfg +4 -0
frbench-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 HKU Trustworthy AI and Systems Research (TASR) Lab
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
frbench-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,347 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: frbench
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Unified, differentiable face-recognition module for AFR and FR research
|
|
5
|
+
Author: Ziling Wang
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/HKU-TASR/FRBench
|
|
8
|
+
Project-URL: Repository, https://github.com/HKU-TASR/FRBench
|
|
9
|
+
Keywords: face-recognition,anti-facial-recognition,pytorch,embeddings
|
|
10
|
+
Classifier: Development Status :: 4 - Beta
|
|
11
|
+
Classifier: Intended Audience :: Science/Research
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
17
|
+
Requires-Python: >=3.10
|
|
18
|
+
Description-Content-Type: text/markdown
|
|
19
|
+
License-File: LICENSE
|
|
20
|
+
Requires-Dist: torch
|
|
21
|
+
Requires-Dist: torchvision
|
|
22
|
+
Requires-Dist: pyyaml
|
|
23
|
+
Requires-Dist: tqdm
|
|
24
|
+
Provides-Extra: demo
|
|
25
|
+
Requires-Dist: pillow; extra == "demo"
|
|
26
|
+
Requires-Dist: matplotlib; extra == "demo"
|
|
27
|
+
Requires-Dist: seaborn; extra == "demo"
|
|
28
|
+
Dynamic: license-file
|
|
29
|
+
|
|
30
|
+
# FRBench
|
|
31
|
+
|
|
32
|
+
An easy-to-use, **end-to-end differentiable** face-recognition module that ships **45+** pretrained weights,
|
|
33
|
+
|
|
34
|
+
- covering **25 backbones**, **9 loss functions**, and **3 training datasets** of different **types**,
|
|
35
|
+
- spanning **generations** from **2015 to 2024**.
|
|
36
|
+
|
|
37
|
+
It is designed for **anti-facial-recognition (AFR)** research, where the transferability of attacks across backbones, loss functions, and training datasets is a central concern. But we believe that it benefits the face recognition field just as much.
|
|
38
|
+
|
|
39
|
+
Hand it any RGB image tensor (aligned or not); it detects and aligns the face(s), runs the chosen pretrained backbone, and returns embeddings ready for cosine similarity. Because the whole pipeline (detect → crop → align → backbone) is written in pure, differentiable PyTorch, **gradients flow from the embeddings all the way back to the input pixels**, enabling adversarial attacks and other gradient-based methods.
|
|
40
|
+
|
|
41
|
+
## Table of Contents
|
|
42
|
+
|
|
43
|
+
- [Highlights](#highlights)
|
|
44
|
+
- [Motivation](#motivation)
|
|
45
|
+
- [Installation](#installation)
|
|
46
|
+
- [Usage](#usage)
|
|
47
|
+
- [Model Zoo](#model-zoo)
|
|
48
|
+
- [Acknowledgement](#acknowledgement)
|
|
49
|
+
- [License](#license)
|
|
50
|
+
- [Citation](#citation)
|
|
51
|
+
|
|
52
|
+
## Highlights
|
|
53
|
+
|
|
54
|
+
- **Unified & consistent.** One module, one API, one weight format for 45+ models — no more stitching together incompatible repos.
|
|
55
|
+
- **Differentiable end-to-end.** Detection is non-differentiable, but cropping and 5-point alignment use `grid_sample` and a closed-form similarity transform, so the embedding is differentiable w.r.t. the raw input image.
|
|
56
|
+
|
|
57
|
+
## Motivation
|
|
58
|
+
|
|
59
|
+
In anti-facial-recognition (AFR) research, one needs to understand how well an attack transfers across face-recognition backbones, loss functions, and training datasets of different **types** and **generations**. For instance, training an attack on `ResNet50` and evaluating it on `SwinV2-T` (same loss, same data) fairly measures generalization to a next-generation architecture; training on `IRSE50` and evaluating on `IRSE100` isolates the effect of model size.
|
|
60
|
+
|
|
61
|
+
However, existing model zoos such as [face.evoLVe](https://github.com/ZhaoJ9014/face.evoLVe) and [FaceX-Zoo](https://github.com/JDAI-CV/FaceX-Zoo) do not offer this controlled variation over backbones, losses, and datasets. Researchers end up collecting and deploying models from many libraries, and still cannot be confident that the "Swin Transformer" in repo A is the same as the "SwinV1" in repo B.
|
|
62
|
+
|
|
63
|
+
FRBench addresses this with a **single, unified, differentiable, easy-to-use PyTorch module** backed by a rich set of consistently-trained pretrained weights. (The training code may be released later.)
|
|
64
|
+
|
|
65
|
+
## Quickstart
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
import frbench
|
|
69
|
+
import torch
|
|
70
|
+
|
|
71
|
+
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
|
72
|
+
|
|
73
|
+
# Inputs must be RGB float tensors in [0, 255] — NOT torchvision ToTensor() [0, 1].
|
|
74
|
+
img = torch.rand(3, 480, 640, device=device) * 255.0
|
|
75
|
+
|
|
76
|
+
fr = frbench.FR("mobilevitv3-s", "arcface", "ms1m").to(device)
|
|
77
|
+
result = fr(img, l2_normalize=True) # FREmbedResult(embeddings, indices, crops)
|
|
78
|
+
print(result.embeddings.shape) # (1, 512) when one face is found
|
|
79
|
+
|
|
80
|
+
# Discover available models programmatically
|
|
81
|
+
for m in frbench.list_models():
|
|
82
|
+
print(m.backbone, m.loss, m.dataset)
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Install with `pip install frbench` (or `pip install "frbench[demo]"` for the notebook extras).
|
|
86
|
+
|
|
87
|
+
## Installation
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
conda create -n ptfr python=3.12
|
|
91
|
+
conda activate ptfr
|
|
92
|
+
pip install frbench # core: torch, torchvision, pyyaml, tqdm
|
|
93
|
+
pip install "frbench[demo]" # optional: pillow, matplotlib, seaborn
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The core module needs `torch`, `torchvision`, `pyyaml`, and `tqdm` (for download progress bars). The demo notebook additionally uses `pillow`, `matplotlib`, and `seaborn`.
|
|
97
|
+
|
|
98
|
+
For local development, clone the repository and run `pip install -e ".[demo]"`.
|
|
99
|
+
|
|
100
|
+
Weights are downloaded on demand into `~/.frbench` (override with `FRBENCH_CACHE`). Prefetch assets with the CLI:
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
frbench-download --list # show all manifest keys
|
|
104
|
+
frbench-download --list-models # FR models only (no detectors)
|
|
105
|
+
frbench-download mobilefacenet_arcface_ms1m # download one model
|
|
106
|
+
frbench-download --all # download everything
|
|
107
|
+
frbench-download --all --quiet # no progress / logs
|
|
108
|
+
frbench-download --refresh --list # re-fetch manifest, then list
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
The public weights are downloaded directly from GitHub Releases; no GitHub login or token is required.
|
|
112
|
+
|
|
113
|
+
Progress bars use ``tqdm`` and are controlled by `set_download_verbose(bool)` (or `--quiet` on the download script); see [Notes on verbosity](#notes-on-verbosity).
|
|
114
|
+
|
|
115
|
+
## Usage
|
|
116
|
+
|
|
117
|
+
**Input contract:** pass RGB **float** tensors in **`[0, 255]`** (not `torchvision` `ToTensor()` `[0, 1]`). The preprocessor warns if values look normalized.
|
|
118
|
+
|
|
119
|
+
See [the demo notebook](./demo.ipynb) for a complete, runnable walkthrough.
|
|
120
|
+
|
|
121
|
+
`forward` / `embed` return a `frbench.FREmbedResult` named tuple with fields `embeddings`, `indices`, and `crops`. When no faces are found, `embeddings` and `crops` are **empty tensors** (never `None`).
|
|
122
|
+
|
|
123
|
+
Key options (see the docstring for the full list):
|
|
124
|
+
|
|
125
|
+
- `need_crop=False` — inputs are already-aligned 112×112 crops; skip detection.
|
|
126
|
+
- `need_align=False` — crop a (loosened, see `loosen_crop`) square box instead of 5-point aligning.
|
|
127
|
+
- `keep_largest=False` — return an embedding for every detected face, not just the largest.
|
|
128
|
+
- `discard_invalid=True` — drop images with no detected face (default: fall back to the whole image).
|
|
129
|
+
- `tta=("flip_horizontal",)` — test-time augmentations to average over (**default: horizontal flip; doubles backbone cost**). Pass `tta=()` to disable.
|
|
130
|
+
- `l2_normalize=True` — L2-normalize the returned embeddings for direct cosine comparison.
|
|
131
|
+
- `eager_load=True` (constructor) — download weights and build the backbone at init instead of first use.
|
|
132
|
+
- `detections=` — pass precomputed detections from `FR.detect()` to skip re-detection during iterative attacks.
|
|
133
|
+
|
|
134
|
+
```python
|
|
135
|
+
# Detect once, then reuse detections in an optimization loop
|
|
136
|
+
dets = fr.detect([img1, img2])
|
|
137
|
+
for step in range(100):
|
|
138
|
+
result = fr([img1, img2], detections=dets)
|
|
139
|
+
loss = ...
|
|
140
|
+
loss.backward()
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
### Notes on devices
|
|
144
|
+
|
|
145
|
+
Inference runs natively on CUDA, MPS, and CPU. For **backprop on Apple MPS**, PyTorch needs a CPU fallback for a couple of ops (e.g. `grid_sample`'s backward); the package sets `PYTORCH_ENABLE_MPS_FALLBACK=1` on import, which is honored as long as the package is imported **before** `torch`. If you import `torch` first, set it yourself beforehand:
|
|
146
|
+
|
|
147
|
+
```python
|
|
148
|
+
import os; os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
|
|
149
|
+
import torch
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
### Notes on configuration
|
|
153
|
+
|
|
154
|
+
Settings can be overridden via environment variables or module constants in [`frbench/_config.py`](./frbench/_config.py):
|
|
155
|
+
|
|
156
|
+
| Setting | Env var | Default |
|
|
157
|
+
| --- | --- | --- |
|
|
158
|
+
| Cache directory | `FRBENCH_CACHE` | `~/.frbench` |
|
|
159
|
+
| GitHub repo | `FRBENCH_REPO` | `HKU-TASR/FRBench` |
|
|
160
|
+
| Release tag | `FRBENCH_RELEASE` | `weights-v1.0.0` |
|
|
161
|
+
|
|
162
|
+
Module-level `frbench.CACHE`, `frbench.REPO`, and `frbench.RELEASE` are also available.
|
|
163
|
+
|
|
164
|
+
The face detector defaults to `retinaface_mobilenetv1` and can be changed per model via `FR(..., detector="retinaface_resnet50")`.
|
|
165
|
+
|
|
166
|
+
### Notes on verbosity
|
|
167
|
+
|
|
168
|
+
Output has two independent switches, both enabled by default and toggleable on the fly:
|
|
169
|
+
|
|
170
|
+
```python
|
|
171
|
+
import frbench
|
|
172
|
+
|
|
173
|
+
frbench.set_warnings(False) # silence inference warnings (e.g. no face detected)
|
|
174
|
+
frbench.set_download_verbose(False) # silence download status + progress bars
|
|
175
|
+
frbench.set_verbose(False) # convenience: both at once
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
- **Inference warnings** (no face detected, unsupported TTA, etc.) go through Python's standard `warnings` module under the `frbench.FRBenchWarning` category, so you can also filter or capture them with the usual tools, e.g. `warnings.filterwarnings("ignore", category=frbench.FRBenchWarning)` or `with warnings.catch_warnings(record=True) as w: ...`.
|
|
179
|
+
- **Download verbosity** (log messages and the `tqdm` progress bar) is a separate channel controlled by `set_download_verbose` (or `--quiet` on `frbench-download`).
|
|
180
|
+
- **Critical problems** (invalid input tensors, missing manifest keys, failed downloads) raise exceptions (`ValueError`, `frbench.FRBenchDownloadError`, etc.) regardless of the switches above.
|
|
181
|
+
|
|
182
|
+
### Notes on update checks
|
|
183
|
+
|
|
184
|
+
On the first `FR(...)` construction or `frbench-download` command in a process, FRBench checks whether a newer package or weights release is available. Results are cached for 24 hours, network failures are ignored, and no check runs merely from importing the package.
|
|
185
|
+
|
|
186
|
+
Set `FRBENCH_NO_UPDATE_CHECK=1` or call `frbench.set_update_check(False)` to disable these reminders. Installed versions remain pinned to the weights release they were tested with; updating is always explicit.
|
|
187
|
+
|
|
188
|
+
## Model Zoo
|
|
189
|
+
|
|
190
|
+
Every model is addressed by the same three manifest keys you pass to `FR(backbone_type, loss_type, dataset_type)`. All accuracies below are in percent.
|
|
191
|
+
|
|
192
|
+
| Name | `backbone_type` | `loss_type` | `dataset_type` | LFW | CPLFW | CALFW | CFP-FP | AgeDB-30 | MegaFace (1M; Rank-1 Id) | MegaFace (1M; TAR@FAR=1e-4 Ver) | IJB-C (N1D1F1; TAR@FAR=1e-4 Ver) |
|
|
193
|
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
|
194
|
+
| ResNet100-ArcFace-MS1M | `resnet-100` | `arcface` | `ms1m` | 99.75 | 92.18 | 95.73 | 97.3 | 96.37 | 93.19 | 98.49 | 95.21 |
|
|
195
|
+
| ResNetV2_100-ArcFace-MS1M | `resnetv2-100` | `arcface` | `ms1m` | 99.83 | 93.6 | 95.77 | 98.29 | 97.2 | 96.72 | 99.26 | 95.5 |
|
|
196
|
+
| DenseNet201-ArcFace-MS1M | `densenet-201` | `arcface` | `ms1m` | 99.85 | 93.42 | 96 | 98.56 | 97.92 | 97.75 | 99.38 | 95.91 |
|
|
197
|
+
| IRSE100-ArcFace-MS1M | `irse-100` | `arcface` | `ms1m` | 99.77 | 93.08 | 95.87 | 98.3 | 97.7 | 97.48 | 99.61 | 96.31 |
|
|
198
|
+
| EfficientNet_b4-ArcFace-MS1M | `efficientnetv1-b4` | `arcface` | `ms1m` | 99.73 | 91.83 | 95.72 | 97.33 | 96.95 | 95.89 | 99.08 | 94.98 |
|
|
199
|
+
| MobileFaceNet_ECA-ArcFace-MS1M | `mobilefacenet` | `arcface` | `ms1m` | 99.68 | 90.85 | 95.67 | 95.2 | 96.82 | 93.93 | 98.41 | 93.23 |
|
|
200
|
+
| SwinV1_B-ArcFace-MS1M | `swinv1-b` | `arcface` | `ms1m` | 99.8 | 93.7 | 96.05 | 98.64 | 98.03 | 98.8 | 99.56 | 96.8 |
|
|
201
|
+
| ConvNeXt_B-ArcFace-MS1M | `convnext-b` | `arcface` | `ms1m` | 99.83 | 93.7 | 96.02 | 98.79 | 97.75 | 98.61 | 99.49 | 96.35 |
|
|
202
|
+
| ConvNeXtV2_B-ArcFace-MS1M | `convnextv2-b` | `arcface` | `ms1m` | 99.8 | 93.82 | 96.03 | 98.8 | 97.65 | 98.63 | 99.51 | 96.41 |
|
|
203
|
+
| SwinMLP_B-ArcFace-MS1M | `swinmlp-b` | `arcface` | `ms1m` | 99.82 | 93.27 | 96.22 | 98.69 | 97.95 | 98.86 | 99.55 | 96.78 |
|
|
204
|
+
| SwinV2_B-ArcFace-MS1M | `swinv2-b` | `arcface` | `ms1m` | 99.82 | 93.48 | 96.25 | 98.69 | 97.88 | 98.67 | 99.54 | 96.49 |
|
|
205
|
+
| MobileViTV1_S-ArcFace-MS1M | `mobilevit-s` | `arcface` | `ms1m` | 99.5 | 90.47 | 95.43 | 94.71 | 95.67 | 90.83 | 97.74 | 92.82 |
|
|
206
|
+
| MobileViTV2_2.0-ArcFace-MS1M | `mobilevitv2-2.0` | `arcface` | `ms1m` | 99.5 | 91.03 | 95.58 | 96.24 | 96.23 | 94.62 | 98.69 | 93.97 |
|
|
207
|
+
| MobileViTV3V1_S-ArcFace-MS1M | `mobilevitv3-s` | `arcface` | `ms1m` | 99.68 | 90.95 | 95.53 | 95.21 | 96.27 | 92.22 | 98.08 | 93.4 |
|
|
208
|
+
| MobileViTV3V2_2.0-ArcFace-MS1M | `mobilevitv3-2.0` | `arcface` | `ms1m` | 99.75 | 91.8 | 95.7 | 96.87 | 96.67 | 95.79 | 98.97 | 94.48 |
|
|
209
|
+
| MobileNetV1_W1-ArcFace-MS1M | `mobilenet-w1` | `arcface` | `ms1m` | 99.58 | 90.7 | 95.8 | 95.96 | 96.97 | 95.02 | 98.64 | 93.6 |
|
|
210
|
+
| MobileNetV2_W1-ArcFace-MS1M | `mobilenetv2-w1` | `arcface` | `ms1m` | 99.52 | 89.72 | 95.5 | 93.73 | 96.77 | 92.48 | 97.96 | 92.82 |
|
|
211
|
+
| MobileNetV3_L-ArcFace-MS1M | `mobilenetv3-l` | `arcface` | `ms1m` | 99.57 | 90.15 | 95.7 | 94.71 | 96.98 | 93.65 | 98.3 | 92.88 |
|
|
212
|
+
| MobileNetV4_Conv_M-ArcFace-MS1M | `mobilenetv4conv-m` | `arcface` | `ms1m` | 99.72 | 91.63 | 96.03 | 96.74 | 97.38 | 96 | 99.02 | 94.63 |
|
|
213
|
+
| SwinV2_T-ArcFace-MS1M | `swinv2-t` | `arcface` | `ms1m` | 99.78 | 92.67 | 96.05 | 97.99 | 97.58 | 97.92 | 99.42 | 96 |
|
|
214
|
+
| SwinV2_S-ArcFace-MS1M | `swinv2-s` | `arcface` | `ms1m` | 99.8 | 93.37 | 95.88 | 98.36 | 97.93 | 98.23 | 99.43 | 96.33 |
|
|
215
|
+
| SwinV2_L-ArcFace-MS1M | `swinv2-l` | `arcface` | `ms1m` | 99.78 | 93.95 | 96.05 | 98.81 | 98.18 | 98.84 | 99.58 | 96.78 |
|
|
216
|
+
| IRSE18-ArcFace-MS1M | `irse-18` | `arcface` | `ms1m` | 99.47 | 91.57 | 95.88 | 96.66 | 96.87 | 94.47 | 98.57 | 94.04 |
|
|
217
|
+
| IRSE34-ArcFace-MS1M | `irse-34` | `arcface` | `ms1m` | 99.8 | 92.78 | 95.95 | 97.93 | 97.72 | 96.92 | 99.24 | 95.63 |
|
|
218
|
+
| IRSE50-ArcFace-MS1M | `irse-50` | `arcface` | `ms1m` | 99.8 | 93.27 | 95.97 | 98.21 | 97.73 | 97.26 | 99.29 | 96.07 |
|
|
219
|
+
| IRSE100-Triplet-MS1M | `irse-100` | `tripletloss` | `ms1m` | 99.78 | 91.97 | 95.47 | 97.93 | 97.1 | 91.05 | 98.27 | 91.17 |
|
|
220
|
+
| IRSE100-Center-MS1M | `irse-100` | `centerloss` | `ms1m` | 99.77 | 92.98 | 95.92 | 98.49 | 97.32 | 96.24 | 99.32 | 93.59 |
|
|
221
|
+
| IRSE100-SphereFace-MS1M | `irse-100` | `sphereface` | `ms1m` | 99.8 | 93.95 | 96.23 | 98.97 | 98.33 | 98.18 | 99.49 | 96.6 |
|
|
222
|
+
| IRSE100-CosFace-MS1M | `irse-100` | `cosface` | `ms1m` | 99.8 | 93.9 | 96.07 | 98.89 | 98.28 | 98.83 | 99.59 | 97 |
|
|
223
|
+
| IRSE100-CurricularFace-MS1M | `irse-100` | `curricularface` | `ms1m` | 99.82 | 93.82 | 95.98 | 98.8 | 98.15 | 98.93 | 99.59 | 96.98 |
|
|
224
|
+
| IRSE100-MagFace-MS1M | `irse-100` | `magface` | `ms1m` | 99.82 | 93.67 | 95.98 | 98.86 | 98.47 | 98.94 | 99.58 | 96.89 |
|
|
225
|
+
| IRSE100-AdaFace-MS1M | `irse-100` | `adaface` | `ms1m` | 99.82 | 93.72 | 96.17 | 98.76 | 98.23 | 98.88 | 99.61 | 96.96 |
|
|
226
|
+
| IRSE100-UniFace-MS1M | `irse-100` | `uniface` | `ms1m` | 99.8 | 93.43 | 96.13 | 98.64 | 98.3 | 98.77 | 99.57 | 96.87 |
|
|
227
|
+
| SwinV2_B-Triplet-MS1M | `swinv2-b` | `tripletloss` | `ms1m` | 99.77 | 93.2 | 95.88 | 98.36 | 97.05 | 94.65 | 98.97 | 93.66 |
|
|
228
|
+
| SwinV2_B-Center-MS1M | `swinv2-b` | `centerloss` | `ms1m` | 99.55 | 89.93 | 93.92 | 97.4 | 95.57 | 93.75 | 97.54 | 82.23 |
|
|
229
|
+
| SwinV2_B-SphereFace-MS1M | `swinv2-b` | `sphereface` | `ms1m` | 99.85 | 93.8 | 96.28 | 98.47 | 97.82 | 97.71 | 99.38 | 96.28 |
|
|
230
|
+
| SwinV2_B-CosFace-MS1M | `swinv2-b` | `cosface` | `ms1m` | 99.82 | 93.8 | 96.25 | 98.63 | 97.8 | 98.36 | 99.49 | 96.63 |
|
|
231
|
+
| SwinV2_B-CurricularFace-MS1M | `swinv2-b` | `curricularface` | `ms1m` | 99.8 | 93.63 | 96.07 | 98.43 | 97.93 | 98.56 | 99.47 | 96.6 |
|
|
232
|
+
| SwinV2_B-MagFace-MS1M | `swinv2-b` | `magface` | `ms1m` | 99.8 | 93.87 | 96.07 | 98.54 | 97.95 | 98.62 | 99.5 | 96.5 |
|
|
233
|
+
| SwinV2_B-AdaFace-MS1M | `swinv2-b` | `adaface` | `ms1m` | 99.82 | 93.62 | 96 | 98.27 | 97.72 | 98.66 | 99.5 | 96.64 |
|
|
234
|
+
| SwinV2_B-UniFace-MS1M | `swinv2-b` | `uniface` | `ms1m` | 99.75 | 93.55 | 95.92 | 98.51 | 97.6 | 98.3 | 99.42 | 96.51 |
|
|
235
|
+
| IRSE100-ArcFace-WebFace4M | `irse-100` | `arcface` | `webface4m` | 99.68 | 92.97 | 95.4 | 98.3 | 96.12 | 95.19 | 98.88 | 95.13 |
|
|
236
|
+
| IRSE100-ArcFace-Glint360k | `irse-100` | `arcface` | `glint360k` | 99.8 | 94.07 | 95.88 | 98.81 | 97.62 | 98.25 | 99.47 | 96.71 |
|
|
237
|
+
| SwinV2_B-ArcFace-WebFace4M | `swinv2-b` | `arcface` | `webface4m` | 99.82 | 94.42 | 95.75 | 98.69 | 97.58 | 97.14 | 99.39 | 96.82 |
|
|
238
|
+
| SwinV2_B-ArcFace-Glint360k | `swinv2-b` | `arcface` | `glint360k` | 99.85 | 95.03 | 96.07 | 99.03 | 98.3 | 98.81 | 99.63 | 97.43 |
|
|
239
|
+
|
|
240
|
+
### Coverage of backbones
|
|
241
|
+
|
|
242
|
+
Under the same loss function (ArcFace) and the same dataset (MS1M), we have:
|
|
243
|
+
|
|
244
|
+
1. SOTA backbones from 2015-2024
|
|
245
|
+
|
|
246
|
+
| Backbone | Year | Venue |
|
|
247
|
+
| --- | --- | --- |
|
|
248
|
+
| [ResNet](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf) | 2015 | NIPS |
|
|
249
|
+
| [ResNetV2](https://arxiv.org/pdf/1603.05027) | 2016 | ECCV |
|
|
250
|
+
| [DenseNet](https://openaccess.thecvf.com/content_cvpr_2017/papers/Huang_Densely_Connected_Convolutional_CVPR_2017_paper.pdf) | 2017 | CVPR |
|
|
251
|
+
| [SE-Net](https://openaccess.thecvf.com/content_cvpr_2018/papers/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.pdf)(IRSE) | 2018 | CVPR |
|
|
252
|
+
| [EfficientNet](https://proceedings.mlr.press/v97/tan19a/tan19a.pdf) | 2019 | ICML |
|
|
253
|
+
| [ECA-Net](https://openaccess.thecvf.com/content_CVPR_2020/papers/Wang_ECA-Net_Efficient_Channel_Attention_for_Deep_Convolutional_Neural_Networks_CVPR_2020_paper.pdf)(MobileFaceNet_ECA) | 2020 | CVPR |
|
|
254
|
+
| [SwinV1](https://openaccess.thecvf.com/content/ICCV2021/papers/Liu_Swin_Transformer_Hierarchical_Vision_Transformer_Using_Shifted_Windows_ICCV_2021_paper.pdf) | 2021 | ICCV |
|
|
255
|
+
| [ConvNeXtV1](https://openaccess.thecvf.com/content/CVPR2022/papers/Liu_A_ConvNet_for_the_2020s_CVPR_2022_paper.pdf) | 2022 | CVPR |
|
|
256
|
+
| [ConvNeXtV2](https://openaccess.thecvf.com/content/CVPR2023/papers/Woo_ConvNeXt_V2_Co-Designing_and_Scaling_ConvNets_With_Masked_Autoencoders_CVPR_2023_paper.pdf) | 2023 | CVPR |
|
|
257
|
+
| [MobileNetV4](https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/05647.pdf) | 2024 | ECCV |
|
|
258
|
+
|
|
259
|
+
2. Backbones of the same family across generations
|
|
260
|
+
|
|
261
|
+
| Backbone | Family | Paradigm |
|
|
262
|
+
| --- | --- | --- |
|
|
263
|
+
| SwinMLP | Swin Transformer | Transformer |
|
|
264
|
+
| SwinV1 | Swin Transformer | Transformer |
|
|
265
|
+
| SwinV2 | Swin Transformer | Transformer |
|
|
266
|
+
| MobileViTV1 | MobileViT | Transformer |
|
|
267
|
+
| MobileViTV2 | MobileViT | Transformer |
|
|
268
|
+
| MobileViTV3V1 | MobileViT | Transformer |
|
|
269
|
+
| MobileViTV3V2 | MobileViT | Transformer |
|
|
270
|
+
| ConvNeXtV1 | ConvNeXt | CNN |
|
|
271
|
+
| ConvNeXtV2 | ConvNeXt | CNN |
|
|
272
|
+
| MobileNetV1 | MobileNet | CNN |
|
|
273
|
+
| MobileNetV2 | MobileNet | CNN |
|
|
274
|
+
| MobileNetV3 | MobileNet | CNN |
|
|
275
|
+
| MobileNetV4-Conv | MobileNet | CNN |
|
|
276
|
+
|
|
277
|
+
3. Backbones of the same type across sizes
|
|
278
|
+
|
|
279
|
+
| Backbone | Type | Parameter Size (M) | Paradigm |
|
|
280
|
+
| --- | --- | --- | --- |
|
|
281
|
+
| SwinV2-T | SwinV2 | 47.099 | Transformer |
|
|
282
|
+
| SwinV2-S | SwinV2 | 68.57 | Transformer |
|
|
283
|
+
| SwinV2-B | SwinV2 | 112.926 | Transformer |
|
|
284
|
+
| SwinV2-L | SwinV2 | 234.078 | Transformer |
|
|
285
|
+
| IRSE18 | IRSE | 24.115 | CNN |
|
|
286
|
+
| IRSE34 | IRSE | 34.303 | CNN |
|
|
287
|
+
| IRSE50 | IRSE | 43.824 | CNN |
|
|
288
|
+
| IRSE100 | IRSE | 65.549 | CNN |
|
|
289
|
+
|
|
290
|
+
### Coverage of losses
|
|
291
|
+
|
|
292
|
+
Under the same backbone (IRSE100 for CNN, SwinV2-B for Transformer) and the same dataset (MS1M), we have SOTA loss functions from 2015-2023
|
|
293
|
+
|
|
294
|
+
| Loss | Year | Venue |
|
|
295
|
+
| --- | --- | --- |
|
|
296
|
+
| [Triplet Loss](https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Schroff_FaceNet_A_Unified_2015_CVPR_paper.pdf) | 2015 | CVPR |
|
|
297
|
+
| [Center Loss](https://kpzhang93.github.io/papers/eccv2016.pdf) | 2016 | ECCV |
|
|
298
|
+
| [SphereFace](https://openaccess.thecvf.com/content_cvpr_2017/papers/Liu_SphereFace_Deep_Hypersphere_CVPR_2017_paper.pdf) | 2017 | CVPR |
|
|
299
|
+
| [CosFace/LMCL](https://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_CosFace_Large_Margin_CVPR_2018_paper.pdf) | 2018 | CVPR |
|
|
300
|
+
| [ArcFace](https://openaccess.thecvf.com/content_CVPR_2019/papers/Deng_ArcFace_Additive_Angular_Margin_Loss_for_Deep_Face_Recognition_CVPR_2019_paper.pdf) | 2019 | CVPR |
|
|
301
|
+
| [CurricularFace](https://openaccess.thecvf.com/content_CVPR_2020/papers/Huang_CurricularFace_Adaptive_Curriculum_Learning_Loss_for_Deep_Face_Recognition_CVPR_2020_paper.pdf) | 2020 | CVPR |
|
|
302
|
+
| [MagFace](https://openaccess.thecvf.com/content/CVPR2021/papers/Meng_MagFace_A_Universal_Representation_for_Face_Recognition_and_Quality_Assessment_CVPR_2021_paper.pdf) | 2021 | CVPR |
|
|
303
|
+
| [AdaFace](https://openaccess.thecvf.com/content/CVPR2022/papers/Kim_AdaFace_Quality_Adaptive_Margin_for_Face_Recognition_CVPR_2022_paper.pdf) | 2022 | CVPR |
|
|
304
|
+
| [UniFace](https://openaccess.thecvf.com/content/ICCV2023/papers/Zhou_UniFace_Unified_Cross-Entropy_Loss_for_Deep_Face_Recognition_ICCV_2023_paper.pdf) | 2023 | ICCV |
|
|
305
|
+
|
|
306
|
+
### Coverage of datasets
|
|
307
|
+
|
|
308
|
+
Under the same backbone (IRSE100 for CNN, SwinV2-B for Transformer) and the same loss function (ArcFace), we have 3 datasets of varying image and identity counts:
|
|
309
|
+
|
|
310
|
+
| Dataset | Images | Identities |
|
|
311
|
+
| --- | --- | --- |
|
|
312
|
+
| [MS1M](https://github.com/ZhaoJ9014/face.evoLVe) | 5,822,653 | 85,742 |
|
|
313
|
+
| [WebFace4M](https://github.com/HaiyuWu/vec2face) | 4,235,242 | 205,990 |
|
|
314
|
+
| [Glint360k](https://github.com/deepinsight/insightface) | 17,091,657 | 360,232 |
|
|
315
|
+
|
|
316
|
+
## Acknowledgement
|
|
317
|
+
|
|
318
|
+
Codes in [`frbench/backbones/`](./frbench/backbones/) and [frbench/utils/retinaface.py](./frbench/utils/retinaface.py) are adapted from existing open-source projects:
|
|
319
|
+
|
|
320
|
+
- [timm](https://github.com/huggingface/pytorch-image-models)
|
|
321
|
+
- [cavaface](https://github.com/cavalleria/cavaface)
|
|
322
|
+
- [FaceX-Zoo](https://github.com/JDAI-CV/FaceX-Zoo)
|
|
323
|
+
- [InsightFace](https://github.com/deepinsight/insightface)
|
|
324
|
+
- [facebookresearch/ConvNeXt](https://github.com/facebookresearch/ConvNeXt)
|
|
325
|
+
- [facebookresearch/ConvNeXt-V2](https://github.com/facebookresearch/ConvNeXt-V2)
|
|
326
|
+
- [apple/ml-cvnets](https://github.com/apple/ml-cvnets)
|
|
327
|
+
- [micronDLA/MobileViTv3](https://github.com/micronDLA/MobileViTv3)
|
|
328
|
+
- [jaiwei98/mobile-vit-pytorch](https://github.com/jaiwei98/mobile-vit-pytorch)
|
|
329
|
+
- [microsoft/Swin-Transformer](https://github.com/microsoft/Swin-Transformer)
|
|
330
|
+
- [HKU-TASR/Protego](https://github.com/HKU-TASR/Protego)
|
|
331
|
+
|
|
332
|
+
## License
|
|
333
|
+
|
|
334
|
+
This project is licensed under the [MIT License](./LICENSE).
|
|
335
|
+
|
|
336
|
+
## Citation
|
|
337
|
+
|
|
338
|
+
If you find our project useful in your research, please consider citing:
|
|
339
|
+
|
|
340
|
+
```bibtex
|
|
341
|
+
@inproceedings{wang2026protego,
|
|
342
|
+
title={Protego: User-Centric Pose-Invariant Privacy Protection Against Face Recognition-Induced Digital Footprint Exposure},
|
|
343
|
+
author={Ziling Wang and Shuya Yang and Jialin Lu and Ka-Ho Chow},
|
|
344
|
+
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
|
|
345
|
+
year={2026}
|
|
346
|
+
}
|
|
347
|
+
```
|