adv-lib 0.2.2__tar.gz → 0.2.6__tar.gz

{adv_lib-0.2.2 → adv_lib-0.2.6}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.4
 Name: adv-lib
-Version: 0.2.2
+Version: 0.2.6
 Summary: Library of various adversarial attacks resources in PyTorch
 Author-email: Jerome Rony <jerome.rony@gmail.com>
 License: BSD 3-Clause License
@@ -49,6 +49,7 @@ Requires-Dist: visdom>=0.1.8
 Provides-Extra: test
 Requires-Dist: scikit-image; extra == "test"
 Requires-Dist: pytest; extra == "test"
+Dynamic: license-file
 
 
 [![DOI](https://zenodo.org/badge/315504148.svg)](https://zenodo.org/badge/latestdoi/315504148)
@@ -111,6 +112,7 @@ Currently the following classification attacks are implemented in the `adv_lib.a
 
 | Name                                                                                    | Knowledge | Type    | Distance(s)                                               | ArXiv Link                                                                                           |
 |-----------------------------------------------------------------------------------------|-----------|---------|-----------------------------------------------------------|------------------------------------------------------------------------------------------------------|
+| DeepFool (DF)                                                                           | White-box | Minimal | $\ell_2$, $\ell_\infty$                                   | [1511.04599](https://arxiv.org/abs/1511.04599)                                                       |
 | Carlini and Wagner (C&W)                                                                | White-box | Minimal | $\ell_2$, $\ell_\infty$                                   | [1608.04644](https://arxiv.org/abs/1608.04644)                                                       |
 | Projected Gradient Descent (PGD)                                                        | White-box | Budget  | $\ell_\infty$                                             | [1706.06083](https://arxiv.org/abs/1706.06083)                                                       |
 | Structured Adversarial Attack (StrAttack)                                               | White-box | Minimal | $\ell_2$ + group-sparsity                                 | [1808.01664](https://arxiv.org/abs/1808.01664)                                                       |
@@ -123,6 +125,7 @@ Currently the following classification attacks are implemented in the `adv_lib.a
 | Folded Gaussian Attack (FGA)<br /> Voting Folded Gaussian Attack (VFGA)                 | White-box | Minimal | $\ell_0$                                                  | [2011.12423](https://arxiv.org/abs/2011.12423)                                                       |
 | Fast Minimum-Norm (FMN)                                                                 | White-box | Minimal | $\ell_0$, $\ell_1$, $\ell_2$, $\ell_\infty$               | [2102.12827](https://arxiv.org/abs/2102.12827)                                                       |
 | Primal-Dual Gradient Descent (PDGD)<br /> Primal-Dual Proximal Gradient Descent (PDPGD) | White-box | Minimal | $\ell_2$<br />$\ell_0$, $\ell_1$, $\ell_2$, $\ell_\infty$ | [2106.01538](https://arxiv.org/abs/2106.01538)                                                       |
+| SuperDeepFool (SDF)                                                                     | White-box | Minimal | $\ell_2$                                                  | [2303.12481](https://arxiv.org/abs/2303.12481)                                                       |
 | σ-zero                                                                                  | White-box | Minimal | $\ell_0$                                                  | [2402.01879](https://arxiv.org/abs/2402.01879)                                                       |
 
 **Bold** means that this repository contains the official implementation.

{adv_lib-0.2.2 → adv_lib-0.2.6}/README.md

@@ -59,6 +59,7 @@ Currently the following classification attacks are implemented in the `adv_lib.a
 
 | Name                                                                                    | Knowledge | Type    | Distance(s)                                               | ArXiv Link                                                                                           |
 |-----------------------------------------------------------------------------------------|-----------|---------|-----------------------------------------------------------|------------------------------------------------------------------------------------------------------|
+| DeepFool (DF)                                                                           | White-box | Minimal | $\ell_2$, $\ell_\infty$                                   | [1511.04599](https://arxiv.org/abs/1511.04599)                                                       |
 | Carlini and Wagner (C&W)                                                                | White-box | Minimal | $\ell_2$, $\ell_\infty$                                   | [1608.04644](https://arxiv.org/abs/1608.04644)                                                       |
 | Projected Gradient Descent (PGD)                                                        | White-box | Budget  | $\ell_\infty$                                             | [1706.06083](https://arxiv.org/abs/1706.06083)                                                       |
 | Structured Adversarial Attack (StrAttack)                                               | White-box | Minimal | $\ell_2$ + group-sparsity                                 | [1808.01664](https://arxiv.org/abs/1808.01664)                                                       |
@@ -71,6 +72,7 @@ Currently the following classification attacks are implemented in the `adv_lib.a
 | Folded Gaussian Attack (FGA)<br /> Voting Folded Gaussian Attack (VFGA)                 | White-box | Minimal | $\ell_0$                                                  | [2011.12423](https://arxiv.org/abs/2011.12423)                                                       |
 | Fast Minimum-Norm (FMN)                                                                 | White-box | Minimal | $\ell_0$, $\ell_1$, $\ell_2$, $\ell_\infty$               | [2102.12827](https://arxiv.org/abs/2102.12827)                                                       |
 | Primal-Dual Gradient Descent (PDGD)<br /> Primal-Dual Proximal Gradient Descent (PDPGD) | White-box | Minimal | $\ell_2$<br />$\ell_0$, $\ell_1$, $\ell_2$, $\ell_\infty$ | [2106.01538](https://arxiv.org/abs/2106.01538)                                                       |
+| SuperDeepFool (SDF)                                                                     | White-box | Minimal | $\ell_2$                                                  | [2303.12481](https://arxiv.org/abs/2303.12481)                                                       |
 | σ-zero                                                                                  | White-box | Minimal | $\ell_0$                                                  | [2402.01879](https://arxiv.org/abs/2402.01879)                                                       |
 
 **Bold** means that this repository contains the official implementation.

adv_lib-0.2.6/adv_lib/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.2.6"

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/__init__.py

@@ -2,6 +2,7 @@ from .augmented_lagrangian import alma
 from .auto_pgd import apgd, apgd_targeted
 from .carlini_wagner import carlini_wagner_l2, carlini_wagner_linf
 from .decoupled_direction_norm import ddn
+from .deepfool import df
 from .fast_adaptive_boundary import fab
 from .fast_minimum_norm import fmn
 from .perceptual_color_attacks import perc_al
@@ -10,4 +11,5 @@ from .projected_gradient_descent import pgd_linf
 from .sigma_zero import sigma_zero
 from .stochastic_sparse_attacks import fga, vfga
 from .structured_adversarial_attack import str_attack
+from .superdeepfool import sdf
 from .trust_region import tr

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/carlini_wagner/l2.py

@@ -1,9 +1,9 @@
 # Adapted from https://github.com/carlini/nn_robust_attacks
-
-from typing import Tuple, Optional
+import math
+from typing import Optional
 
 import torch
-from torch import nn, optim, Tensor
+from torch import nn, Tensor
 from torch.autograd import grad
 
 from adv_lib.utils.losses import difference_of_logits
@@ -20,6 +20,8 @@ def carlini_wagner_l2(model: nn.Module,
                       binary_search_steps: int = 9,
                       max_iterations: int = 10000,
                       abort_early: bool = True,
+                      β_1: float = 0.9,
+                      β_2: float = 0.999,
                       callback: Optional[VisdomLogger] = None) -> Tensor:
     """
     Carlini and Wagner L2 attack from https://arxiv.org/abs/1608.04644.
@@ -50,6 +52,8 @@ def carlini_wagner_l2(model: nn.Module,
         learning rate and will produce poor results.
     abort_early : bool
         If true, allows early aborts if gradient descent gets stuck.
+    β_1, β_2: float
+        Adam exponential averages smoothing parameters.
     callback : Optional
 
     Returns
@@ -62,25 +66,32 @@ def carlini_wagner_l2(model: nn.Module,
     batch_size = len(inputs)
     batch_view = lambda tensor: tensor.view(batch_size, *[1] * (inputs.ndim - 1))
     t_inputs = (inputs * 2).sub_(1).mul_(1 - 1e-6).atanh_()
-    multiplier = -1 if targeted else 1
+    if not targeted:  # gradient descent if untargeted, else ascent
+        learning_rate *= -1
 
     # set the lower and upper bounds accordingly
     c = torch.full((batch_size,), initial_const, device=device)
     lower_bound = torch.zeros_like(c)
     upper_bound = torch.full_like(c, 1e10)
 
+    # Adam variables
+    modifier = torch.zeros_like(inputs, requires_grad=True)
+    exp_avg = torch.zeros_like(inputs)
+    exp_avg_sq = torch.zeros_like(inputs)
+
     o_best_l2 = torch.full_like(c, float('inf'))
     o_best_adv = inputs.clone()
-    o_adv_found = torch.zeros(batch_size, device=device, dtype=torch.bool)
+    o_adv_found = torch.zeros_like(c, dtype=torch.bool)
 
     i_total = 0
     for outer_step in range(binary_search_steps):
 
         # setup the modifier variable and the optimizer
-        modifier = torch.zeros_like(inputs, requires_grad=True)
-        optimizer = optim.Adam([modifier], lr=learning_rate)
+        nn.init.zeros_(modifier)
+        nn.init.zeros_(exp_avg)
+        nn.init.zeros_(exp_avg_sq)
         best_l2 = torch.full_like(c, float('inf'))
-        adv_found = torch.zeros(batch_size, device=device, dtype=torch.bool)
+        adv_found = torch.zeros_like(o_adv_found)
 
         # The last iteration (if we run many steps) repeat the search once.
         if (binary_search_steps >= 10) and outer_step == (binary_search_steps - 1):
@@ -115,7 +126,7 @@ def carlini_wagner_l2(model: nn.Module,
             o_adv_found.logical_or_(is_both)
             o_best_adv = torch.where(batch_view(o_is_both), adv_inputs.detach(), o_best_adv)
 
-            logit_dists = multiplier * difference_of_logits(logits, labels, labels_infhot=labels_infhot)
+            logit_dists = difference_of_logits(logits, labels, labels_infhot=labels_infhot)
             loss = l2_squared + c * (logit_dists + confidence).clamp_(min=0)
 
             # check if we should abort search if we're getting nowhere.
@@ -124,9 +135,13 @@ def carlini_wagner_l2(model: nn.Module,
                     break
                 prev = loss.detach()
 
-            optimizer.zero_grad(set_to_none=True)
-            modifier.grad = grad(loss.sum(), modifier, only_inputs=True)[0]
-            optimizer.step()
+            g = grad(loss.sum(), modifier, only_inputs=True)[0]
+            exp_avg.mul_(β_1).add_(g, alpha=1 - β_1)
+            exp_avg_sq.mul_(β_2).addcmul_(g, g, value=1 - β_2)
+            bias_correction1 = 1 - β_1 ** (i + 1)
+            bias_correction2 = 1 - β_2 ** (i + 1)
+            denom = exp_avg_sq.sqrt().div_(math.sqrt(bias_correction2)).add_(1e-8)
+            modifier.data.addcdiv_(exp_avg, denom, value=learning_rate / bias_correction1)
 
             if callback:
                 i_total += 1

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/carlini_wagner/linf.py

@@ -1,9 +1,10 @@
 # Adapted from https://github.com/carlini/nn_robust_attacks
-
-from typing import Tuple, Optional
+import math
+from typing import Optional
 
 import torch
-from torch import nn, optim, Tensor
+from torch import nn, Tensor
+from torch.autograd import grad
 
 from adv_lib.utils.losses import difference_of_logits
 from adv_lib.utils.visdom_logger import VisdomLogger
@@ -21,6 +22,8 @@ def carlini_wagner_linf(model: nn.Module,
                         reduce_const: bool = False,
                         decrease_factor: float = 0.9,
                         abort_early: bool = True,
+                        β_1: float = 0.9,
+                        β_2: float = 0.999,
                         callback: Optional[VisdomLogger] = None) -> Tensor:
     """
     Carlini and Wagner Linf attack from https://arxiv.org/abs/1608.04644.
@@ -52,8 +55,8 @@ def carlini_wagner_linf(model: nn.Module,
         Rate at which τ is decreased. Larger produces better quality results.
     abort_early : bool
         If true, allows early aborts if gradient descent gets stuck.
-    image_constraints : Tuple[float, float]
-        Minimum and maximum pixel values.
+    β_1, β_2: float
+        Adam exponential averages smoothing parameters.
     callback : Optional
 
     Returns
@@ -65,12 +68,13 @@ def carlini_wagner_linf(model: nn.Module,
     device = inputs.device
     batch_size = len(inputs)
     t_inputs = (inputs * 2).sub_(1).mul_(1 - 1e-6).atanh_()
-    multiplier = -1 if targeted else 1
+    if not targeted:
+        learning_rate *= -1
 
     # set modifier and the parameters used in the optimization
     modifier = torch.zeros_like(inputs)
     c = torch.full((batch_size,), initial_const, device=device, dtype=torch.float)
-    τ = torch.ones(batch_size, device=device)
+    τ = torch.ones_like(c)
 
     o_adv_found = torch.zeros_like(c, dtype=torch.bool)
     o_best_linf = torch.ones_like(c)
@@ -89,7 +93,8 @@ def carlini_wagner_linf(model: nn.Module,
 
         # setup the optimizer
         modifier_ = modifier[to_optimize].requires_grad_(True)
-        optimizer = optim.Adam([modifier_], lr=learning_rate)
+        exp_avg = torch.zeros_like(modifier_)
+        exp_avg_sq = torch.zeros_like(modifier_)
         c_, τ_ = c[to_optimize], τ[to_optimize]
 
         adv_found = torch.zeros(len(modifier_), device=device, dtype=torch.bool)
@@ -119,7 +124,7 @@ def carlini_wagner_linf(model: nn.Module,
             best_linf = torch.where(is_both, linf, best_linf)
             best_adv = torch.where(batch_view(is_both), adv_inputs.detach(), best_adv)
 
-            logit_dists = multiplier * difference_of_logits(logits, labels_, labels_infhot=labels_infhot)
+            logit_dists = difference_of_logits(logits, labels_, labels_infhot=labels_infhot)
             linf_loss = (adv_inputs - inputs_).abs_().sub_(batch_view(τ_)).clamp_(min=0).flatten(1).sum(1)
             loss = linf_loss + c_ * logit_dists.clamp_(min=0)
 
@@ -127,9 +132,13 @@ def carlini_wagner_linf(model: nn.Module,
             if abort_early and (loss < 0.0001 * c_).all():
                 break
 
-            optimizer.zero_grad()
-            loss.sum().backward()
-            optimizer.step()
+            g = grad(loss.sum(), modifier_, only_inputs=True)[0]
+            exp_avg.mul_(β_1).add_(g, alpha=1 - β_1)
+            exp_avg_sq.mul_(β_2).addcmul_(g, g, value=1 - β_2)
+            bias_correction1 = 1 - β_1 ** (i + 1)
+            bias_correction2 = 1 - β_2 ** (i + 1)
+            denom = exp_avg_sq.sqrt().div_(math.sqrt(bias_correction2)).add_(1e-8)
+            modifier_.data.addcdiv_(exp_avg, denom, value=learning_rate / bias_correction1)
 
             if callback:
                 callback.accumulate_line('logit_dist', total_iters, logit_dists.mean())

adv_lib-0.2.6/adv_lib/attacks/deepfool.py

@@ -0,0 +1,127 @@
+import warnings
+
+import torch
+from torch import Tensor, nn
+from torch.autograd import grad
+
+from adv_lib.utils.attack_utils import get_all_targets
+
+
+def df(model: nn.Module,
+       inputs: Tensor,
+       labels: Tensor,
+       targeted: bool = False,
+       steps: int = 100,
+       overshoot: float = 0.02,
+       norm: float = 2,
+       return_unsuccessful: bool = False,
+       return_targets: bool = False) -> Tensor:
+    """
+    DeepFool attack from https://arxiv.org/abs/1511.04599. Properly implement parallel sample-wise early-stopping.
+
+    Parameters
+    ----------
+    model : nn.Module
+        Model to attack.
+    inputs : Tensor
+        Inputs to attack. Should be in [0, 1].
+    labels : Tensor
+        Labels corresponding to the inputs if untargeted, else target labels.
+    targeted : bool
+        Whether to perform a targeted attack or not.
+    steps : int
+        Maixmum number of attack steps.
+    overshoot : float
+        Ratio by which to overshoot the boundary estimated from linear model.
+    norm : float
+        Norm to minimize in {2, float('inf')}.
+    return_unsuccessful : bool
+        Whether to return unsuccessful adversarial inputs ; used by SuperDeepFool.
+    return_unsuccessful : bool
+        Whether to return last target labels ; used by SuperDeepFool.
+
+    Returns
+    -------
+    adv_inputs : Tensor
+        Modified inputs to be adversarial to the model.
+
+    """
+    if targeted:
+        warnings.warn('DeepFool attack is untargeted only. Returning inputs.')
+        return inputs
+
+    if inputs.min() < 0 or inputs.max() > 1: raise ValueError('Input values should be in the [0, 1] range.')
+    device = inputs.device
+    batch_size = len(inputs)
+    batch_view = lambda tensor: tensor.view(-1, *[1] * (inputs.ndim - 1))
+
+    # Setup variables
+    adv_inputs = inputs.clone()
+    adv_inputs.requires_grad_(True)
+
+    adv_out = inputs.clone()
+    adv_found = torch.zeros(batch_size, dtype=torch.bool, device=device)
+    if return_targets:
+        targets = labels.clone()
+
+    arange = torch.arange(batch_size, device=device)
+    for i in range(steps):
+
+        logits = model(adv_inputs)
+
+        if i == 0:
+            other_labels = get_all_targets(labels=labels, num_classes=logits.shape[1])
+
+        pred_labels = logits.argmax(dim=1)
+        is_adv = (pred_labels == labels) if targeted else (pred_labels != labels)
+
+        if is_adv.any():
+            adv_not_found = ~adv_found
+            adv_out[adv_not_found] = torch.where(batch_view(is_adv), adv_inputs.detach(), adv_out[adv_not_found])
+            adv_found.masked_scatter_(adv_not_found, is_adv)
+            if is_adv.all():
+                break
+
+            not_adv = ~is_adv
+            logits, labels, other_labels = logits[not_adv], labels[not_adv], other_labels[not_adv]
+            arange = torch.arange(not_adv.sum(), device=device)
+
+        f_prime = logits.gather(dim=1, index=other_labels) - logits.gather(dim=1, index=labels.unsqueeze(1))
+        w_prime = []
+        for j, f_prime_k in enumerate(f_prime.unbind(dim=1)):
+            w_prime_k = grad(f_prime_k.sum(), inputs=adv_inputs, retain_graph=(j + 1) < f_prime.shape[1],
+                             only_inputs=True)[0]
+            w_prime.append(w_prime_k)
+        w_prime = torch.stack(w_prime, dim=1)  # batch_size × num_classes × ...
+
+        if is_adv.any():
+            not_adv = ~is_adv
+            adv_inputs, w_prime = adv_inputs[not_adv], w_prime[not_adv]
+
+        if norm == 2:
+            w_prime_norms = w_prime.flatten(2).norm(p=2, dim=2).clamp_(min=1e-6)
+        elif norm == float('inf'):
+            w_prime_norms = w_prime.flatten(2).norm(p=1, dim=2).clamp_(min=1e-6)
+
+        distance = f_prime.detach().abs_().div_(w_prime_norms).add_(1e-4)
+        l_hat = distance.argmin(dim=1)
+
+        if return_targets:
+            targets[~adv_found] = torch.where(l_hat >= labels, l_hat + 1, l_hat)
+
+        if norm == 2:
+            # 1e-4 added in original implementation
+            scale = distance[arange, l_hat] / w_prime_norms[arange, l_hat]
+            adv_inputs.data.addcmul_(batch_view(scale), w_prime[arange, l_hat], value=1 + overshoot)
+        elif norm == float('inf'):
+            adv_inputs.data.addcmul_(batch_view(distance[arange, l_hat]), w_prime[arange, l_hat].sign(),
+                                     value=1 + overshoot)
+        adv_inputs.data.clamp_(min=0, max=1)
+
+    if return_unsuccessful and not adv_found.all():
+        adv_out[~adv_found] = adv_inputs.detach()
+
+    if return_targets:
+        return adv_out, targets
+
+    return adv_out

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/fast_adaptive_boundary/fast_adaptive_boundary.py

@@ -8,6 +8,7 @@ import torch
 from torch import Tensor, nn
 from torch.autograd import grad
 
+from adv_lib.utils.attack_utils import get_all_targets
 from .projections import projection_l1, projection_l2, projection_linf
 
 
@@ -163,13 +164,7 @@ def _fab(model: nn.Module,
     if targets is not None:
         other_labels = targets.unsqueeze(1)
     else:
-        # generate all other labels
-        n_classes = logits.size(1)
-        other_labels = torch.zeros(len(labels), n_classes - 1, dtype=torch.long, device=device)
-        all_classes = set(range(n_classes))
-        for i in range(len(labels)):
-            diff_labels = list(all_classes.difference({labels[i].item()}))
-            other_labels[i] = torch.tensor(diff_labels, device=device)
+        other_labels = get_all_targets(labels=labels, num_classes=logits.shape[1])
 
     get_df_dg = partial(get_best_diff_logits_grads, model=model, labels=labels, other_labels=other_labels, q=dual_norm)
 

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/fast_minimum_norm.py

@@ -59,8 +59,7 @@ def l1_mid_points(x0: Tensor, x1: Tensor, ε: Tensor) -> Tensor:
 
 
 def l2_mid_points(x0: Tensor, x1: Tensor, ε: Tensor) -> Tensor:
-    ε = ε.unsqueeze(1)
-    return x0.flatten(1).mul(1 - ε).addcmul_(ε, x1.flatten(1)).view_as(x0)
+    return torch.lerp(x0.flatten(1), x1.flatten(1), weight=ε.unsqueeze(1)).view_as(x0)
 
 
 def linf_mid_points(x0: Tensor, x1: Tensor, ε: Tensor) -> Tensor:
@@ -194,24 +193,18 @@ def fmn(model: nn.Module,
                             torch.maximum(ε + 1, (ε * (1 + γ)).floor_()))
             ε.clamp_(min=0)
         else:
-            distance_to_boundary = loss.detach().abs() / δ_grad.flatten(1).norm(p=dual, dim=1).clamp_(min=1e-12)
+            distance_to_boundary = loss.detach().abs_().div_(δ_grad.flatten(1).norm(p=dual, dim=1).clamp_(min=1e-12))
             ε = torch.where(is_adv,
                             torch.minimum(ε * (1 - γ), best_norm),
                             torch.where(adv_found, ε * (1 + γ), δ_norm + distance_to_boundary))
 
         # clip ε
         ε = torch.minimum(ε, worst_norm)
-
-        # normalize gradient
+        # gradient ascent step with normalized gradient
         grad_l2_norms = δ_grad.flatten(1).norm(p=2, dim=1).clamp_(min=1e-12)
-        δ_grad.div_(batch_view(grad_l2_norms))
-
-        # gradient ascent step
-        δ.data.add_(δ_grad, alpha=α)
-
+        δ.data.addcdiv_(δ_grad, batch_view(grad_l2_norms), value=α)
         # project in place
         projection(δ=δ.data, ε=ε)
-
         # clamp
         δ.data.add_(inputs).clamp_(min=0, max=1).sub_(inputs)
 

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/segmentation/alma_prox.py

@@ -10,40 +10,45 @@ from adv_lib.utils.losses import difference_of_logits_ratio
 from adv_lib.utils.visdom_logger import VisdomLogger
 
 
+@torch.compile
 def prox_linf_indicator(δ: Tensor, λ: Tensor, lower: Tensor, upper: Tensor, H: Optional[Tensor] = None,
                         ε: float = 1e-6, section: float = 1 / 3) -> Tensor:
     """Proximity operator of λ||·||_∞ + \iota_Λ in the diagonal metric H. The lower and upper tensors correspond to
     the bounds of Λ. The problem is solved using a ternary search with section 1/3 up to an absolute error of ε on the
     prox. Using a section of 1 - 1/φ (with φ the golden ratio) yields the Golden-section search, which is a bit faster,
     but less numerically stable."""
-    δ_, λ_ = δ.flatten(1), 2 * λ.unsqueeze(1)
-    H_ = H.flatten(1) if H is not None else None
-    δ_proj = δ_.clamp(min=lower.flatten(1), max=upper.flatten(1))
-    right = δ_proj.norm(p=float('inf'), dim=1, keepdim=True)
+    δ_shape = δ.shape
+    δ, λ = δ.flatten(1), 2 * λ
+    H = H.flatten(1) if H is not None else None
+    δ_proj = δ.clamp(min=lower.flatten(1), max=upper.flatten(1))
+    right = δ_proj.norm(p=float('inf'), dim=1)
     left = torch.zeros_like(right)
-    steps = (ε / right.max()).log_().mul_(math.log(math.e, 1 - section)).ceil_().long()
+    steps = (ε / right.max()).log_().div_(math.log(1 - section)).ceil_().long()
     prox, left_third, right_third, f_left, f_right, cond = (None,) * 6
     for _ in range(steps):
         left_third = torch.lerp(left, right, weight=section, out=left_third)
         right_third = torch.lerp(left, right, weight=1 - section, out=right_third)
 
-        prox = torch.clamp(δ_proj, min=-left_third, max=left_third, out=prox).sub_(δ_).square_()
-        if H_ is not None:
-            prox.mul_(H_)
-        f_left = torch.sum(prox, dim=1, keepdim=True, out=f_left)
-        f_left.addcmul_(left_third, λ_)
+        prox = torch.clamp(δ_proj, min=-left_third.unsqueeze(1), max=left_third.unsqueeze(1), out=prox)
+        prox.sub_(δ).square_()
+        if H is not None:
+            prox.mul_(H)
+        f_left = torch.sum(prox, dim=1, out=f_left)
+        f_left.addcmul_(left_third, λ)
 
-        prox = torch.clamp(δ_proj, min=-right_third, max=right_third, out=prox).sub_(δ_).square_()
-        if H_ is not None:
-            prox.mul_(H_)
-        f_right = torch.sum(prox, dim=1, keepdim=True, out=f_right)
-        f_right.addcmul_(right_third, λ_)
+        prox = torch.clamp(δ_proj, min=-right_third.unsqueeze(1), max=right_third.unsqueeze(1), out=prox)
+        prox.sub_(δ).square_()
+        if H is not None:
+            prox.mul_(H)
+        f_right = torch.sum(prox, dim=1, out=f_right)
+        f_right.addcmul_(right_third, λ)
 
         cond = torch.ge(f_left, f_right, out=cond)
         left = torch.where(cond, left_third, left, out=left)
         right = torch.where(cond, right, right_third, out=right)
     left.lerp_(right, weight=0.5)
-    return δ_proj.clamp_(min=-left, max=left).view_as(δ)
+    prox = δ_proj.clamp_(min=-left.unsqueeze(1), max=left.unsqueeze(1)).view(δ_shape)
+    return prox
 
 
 class P(Function):

{adv_lib-0.2.2 → adv_lib-0.2.6}/adv_lib/attacks/stochastic_sparse_attacks.py

@@ -87,7 +87,9 @@ def fga(model: nn.Module,
 
         # add perturbation to inputs
         perturbed_inputs = inputs_.flatten(1).unsqueeze(1).repeat(1, n_samples, 1)
-        perturbed_inputs.scatter_add_(2, i_0.repeat_interleave(n_samples, dim=1).unsqueeze(2), S.unsqueeze(2))
+        perturbed_inputs.scatter_add_(
+            2, i_0.repeat_interleave(n_samples, dim=1, output_size=n_samples).unsqueeze(2), S.unsqueeze(2)
+        )
         perturbed_inputs.clamp_(min=0, max=1)
 
         # get probabilities for perturbed inputs
@@ -199,7 +201,9 @@ def vfga(model: nn.Module,
 
         # add perturbation to inputs
         perturbed_inputs = inputs_.flatten(1).unsqueeze(1).repeat(1, 2 * n_samples, 1)
-        i_plus_minus = torch.cat([i_plus, i_minus], dim=1).repeat_interleave(n_samples, dim=1)
+        i_plus_minus = torch.cat([i_plus, i_minus], dim=1).repeat_interleave(
+            n_samples, dim=1, output_size=2 * n_samples
+        )
         S_plus_minus = torch.cat([S_plus, S_minus], dim=1)
         perturbed_inputs.scatter_add_(2, i_plus_minus.unsqueeze(2), S_plus_minus.unsqueeze(2))
         perturbed_inputs.clamp_(min=0, max=1)

adv_lib-0.2.6/adv_lib/attacks/superdeepfool.py

@@ -0,0 +1,105 @@
+import warnings
+
+import torch
+from torch import Tensor, nn
+from torch.autograd import grad
+
+from .deepfool import df
+
+
+def sdf(model: nn.Module,
+        inputs: Tensor,
+        labels: Tensor,
+        targeted: bool = False,
+        steps: int = 100,
+        df_steps: int = 100,
+        overshoot: float = 0.02,
+        search_iter: int = 10) -> Tensor:
+    """
+    SuperDeepFool attack from https://arxiv.org/abs/2303.12481.
+
+    Parameters
+    ----------
+    model : nn.Module
+        Model to attack.
+    inputs : Tensor
+        Inputs to attack. Should be in [0, 1].
+    labels : Tensor
+        Labels corresponding to the inputs if untargeted, else target labels.
+    targeted : bool
+        Whether to perform a targeted attack or not.
+    steps : int
+        Number of steps.
+    df_steps : int
+        Maximum number of steps for DeepFool attack at each iteration of SuperDeepFool.
+    overshoot : float
+        overshoot parameter in DeepFool.
+    search_iter : int
+        Number of binary search steps at the end of the attack.
+
+    Returns
+    -------
+    adv_inputs : Tensor
+        Modified inputs to be adversarial to the model.
+
+    """
+    if targeted:
+        warnings.warn('DeepFool attack is untargeted only. Returning inputs.')
+        return inputs
+
+    if inputs.min() < 0 or inputs.max() > 1: raise ValueError('Input values should be in the [0, 1] range.')
+    device = inputs.device
+    batch_size = len(inputs)
+    batch_view = lambda tensor: tensor.view(-1, *[1] * (inputs.ndim - 1))
+
+    # Setup variables
+    adv_inputs = inputs_ = inputs
+    labels_ = labels
+    adv_out = inputs.clone()
+    adv_found = torch.zeros(batch_size, dtype=torch.bool, device=device)
+
+    for i in range(steps):
+        logits = model(adv_inputs)
+        pred_labels = logits.argmax(dim=1)
+
+        is_adv = pred_labels != labels_
+        if is_adv.any():
+            adv_not_found = ~adv_found
+            adv_out[adv_not_found] = torch.where(batch_view(is_adv), adv_inputs, adv_out[adv_not_found])
+            adv_found.masked_scatter_(adv_not_found, is_adv)
+            if is_adv.all():
+                break
+
+            not_adv = ~is_adv
+            inputs_, adv_inputs, labels_ = inputs_[not_adv], adv_inputs[not_adv], labels_[not_adv]
+
+        # start by doing deepfool -> need to return adv_inputs even for unsuccessful attacks
+        df_adv_inputs, df_targets = df(model=model, inputs=adv_inputs, labels=labels_, steps=df_steps, norm=2,
+                                       overshoot=overshoot, return_unsuccessful=True, return_targets=True)
+
+        r_df = df_adv_inputs - inputs_
+        df_adv_inputs.requires_grad_(True)
+        logits = model(df_adv_inputs)
+        pred_labels = logits.argmax(dim=1)
+        pred_labels = torch.where(pred_labels != labels_, pred_labels, df_targets)
+
+        logit_diff = logits.gather(1, pred_labels.unsqueeze(1)) - logits.gather(1, labels_.unsqueeze(1))
+        w = grad(logit_diff.sum(), inputs=df_adv_inputs, only_inputs=True)[0]
+        w.div_(batch_view(w.flatten(1).norm(p=2, dim=1).clamp_(min=1e-6)))  # w / ||w||_2
+        scale = torch.linalg.vecdot(r_df.flatten(1), w.flatten(1), dim=1)  # (\tilde{x} - x_0)^T w / ||w||_2
+
+        adv_inputs = adv_inputs.addcmul(batch_view(scale), w)
+        adv_inputs.clamp_(min=0, max=1)  # added compared to original implementation to produce valid adv
+
+    if search_iter:  # binary search to bring perturbation as close to the decision boundary as possible
+        low, high = torch.zeros(batch_size, device=device), torch.ones(batch_size, device=device)
+        for i in range(search_iter):
+            mid = (low + high) / 2
+            logits = torch.lerp(inputs, adv_out, weight=batch_view(mid))
+            pred_labels = model(logits).argmax(dim=1)
+            is_adv = pred_labels != labels
+            high = torch.where(is_adv, mid, high)
+            low = torch.where(is_adv, low, mid)
+        adv_out = torch.lerp(inputs, adv_out, weight=batch_view(high))
+
+    return adv_out