riskcal 0.1.0__tar.gz

riskcal-0.1.0/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2024 Bogdan Kulynych
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

riskcal-0.1.0/PKG-INFO

@@ -0,0 +1,80 @@
+Metadata-Version: 2.1
+Name: riskcal
+Version: 0.1.0
+Summary: Calibrate differentially private algorithms to operational privacy risk measures
+Author: Bogdan Kulynych
+Author-email: bogdan@kulyny.ch
+Requires-Python: >=3.9,<3.13
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: dp-accounting (>=0.4.4,<0.5.0)
+Requires-Dist: scipy (>=1.11.3,<2.0.0)
+Description-Content-Type: text/markdown
+
+## riskcal
+
+[![CI](https://github.com/bogdan-kulynych/riskcal/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bogdan-kulynych/riskcal/actions/workflows/ci.yml)
+[![arXiv](https://img.shields.io/badge/arXiv-2407.02191-b31b1b.svg)](https://arxiv.org/abs/2407.02191)
+
+---
+
+⚠️  This is a research prototype. Avoid or be extra careful when using in production.
+
+---
+
+The library provides tools for calibrating the noise scale in (epsilon, delta)-DP mechanisms to one
+of the two notions of operational attack risk (attack accuracy/advantage, or attack TPR and FPR) instead of the
+(epsilon, delta) parameters, as well as for efficient measurement of these notions.
+The library enables to reduce the noise scale at the same level of targeted attack risk.
+
+
+### Using the Library
+
+Install with:
+```
+pip install riskcal
+```
+
+#### Quickstart
+
+##### Measuring the Exact f-DP / Trade-Off Curve for any DP Mechanism
+To measure the attack trade-off curve (equivalent to attack's receiver-operating curve) for DP-SGD, you can run
+```
+import riskcal
+import numpy as np
+
+alphas = np.array([0.01, 0.05, 0.1])
+betas = riskcal.pld.get_beta(
+    alpha=alphas,
+    noise_multiplier=noise_multiplier,
+    sample_rate=sample_rate,
+    num_steps=num_steps,
+)
+```
+
+You can also get the trade-off curve for any DP mechanism [supported](https://github.com/google/differential-privacy/tree/0b109e959470c43e9f177d5411603b70a56cdc7a/python/dp_accounting)
+by Google's DP accounting library, given its privacy loss distribution (PLD):
+```
+import riskcal
+import numpy as np
+
+alphas = np.array([0.01, 0.05, 0.1])
+betas = riskcal.pld.get_beta_from_pld(pld, alpha=alphas)
+```
+
+##### Calibrating DP-SGD to attack FNR/FPR
+To calibrate noise scale in DP-SGD to a given attack FPR (beta) and FNR (alpha), run:
+```
+import riskcal
+
+noise_multiplier = riskcal.pld.find_noise_multiplier_for_err_rates(
+    beta=0.2,
+    alpha=0.01,
+    sample_rate=sample_rate,
+    num_steps=num_steps
+)
+```
+

riskcal-0.1.0/README.md

@@ -0,0 +1,63 @@
+## riskcal
+
+[![CI](https://github.com/bogdan-kulynych/riskcal/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bogdan-kulynych/riskcal/actions/workflows/ci.yml)
+[![arXiv](https://img.shields.io/badge/arXiv-2407.02191-b31b1b.svg)](https://arxiv.org/abs/2407.02191)
+
+---
+
+⚠️  This is a research prototype. Avoid or be extra careful when using in production.
+
+---
+
+The library provides tools for calibrating the noise scale in (epsilon, delta)-DP mechanisms to one
+of the two notions of operational attack risk (attack accuracy/advantage, or attack TPR and FPR) instead of the
+(epsilon, delta) parameters, as well as for efficient measurement of these notions.
+The library enables to reduce the noise scale at the same level of targeted attack risk.
+
+
+### Using the Library
+
+Install with:
+```
+pip install riskcal
+```
+
+#### Quickstart
+
+##### Measuring the Exact f-DP / Trade-Off Curve for any DP Mechanism
+To measure the attack trade-off curve (equivalent to attack's receiver-operating curve) for DP-SGD, you can run
+```
+import riskcal
+import numpy as np
+
+alphas = np.array([0.01, 0.05, 0.1])
+betas = riskcal.pld.get_beta(
+    alpha=alphas,
+    noise_multiplier=noise_multiplier,
+    sample_rate=sample_rate,
+    num_steps=num_steps,
+)
+```
+
+You can also get the trade-off curve for any DP mechanism [supported](https://github.com/google/differential-privacy/tree/0b109e959470c43e9f177d5411603b70a56cdc7a/python/dp_accounting)
+by Google's DP accounting library, given its privacy loss distribution (PLD):
+```
+import riskcal
+import numpy as np
+
+alphas = np.array([0.01, 0.05, 0.1])
+betas = riskcal.pld.get_beta_from_pld(pld, alpha=alphas)
+```
+
+##### Calibrating DP-SGD to attack FNR/FPR
+To calibrate noise scale in DP-SGD to a given attack FPR (beta) and FNR (alpha), run:
+```
+import riskcal
+
+noise_multiplier = riskcal.pld.find_noise_multiplier_for_err_rates(
+    beta=0.2,
+    alpha=0.01,
+    sample_rate=sample_rate,
+    num_steps=num_steps
+)
+```

riskcal-0.1.0/pyproject.toml

@@ -0,0 +1,40 @@
+[tool.poetry]
+name = "riskcal"
+version = "0.1.0"
+description = "Calibrate differentially private algorithms to operational privacy risk measures"
+authors = ["Bogdan Kulynych <bogdan@kulyny.ch>", "Juan Felipe Gomez <juangomez@g.harvard.edu>"]
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = ">=3.9,<3.13"
+scipy = "^1.11.3"
+dp-accounting = "^0.4.4"
+
+[tool.poetry.group.dev]
+optional = true
+
+[tool.poetry.group.dev.dependencies]
+pytest = "^7.4.2"
+opacus = "^1.4.0"
+pre-commit = "^3.5.0"
+ipdb = "^0.13.13"
+
+[tool.poetry.group.experiments]
+optional = true
+
+[tool.poetry.group.experiments.dependencies]
+seaborn = "^0.13.0"
+pandas = "^2.1.1"
+jupyter = "^1.0.0"
+jupytext = "^1.15.2"
+tqdm = "^4.66.1"
+torch = "==2.0.0"
+prv-accountant = "^0.2.0"
+
+[tool.pytest.ini_options]
+addopts = ["-v"]
+testpaths = ["tests"]
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

riskcal-0.1.0/riskcal/__init__.py

@@ -0,0 +1,3 @@
+from . import blackbox
+from . import utils
+from . import pld

riskcal-0.1.0/riskcal/blackbox.py

@@ -0,0 +1,266 @@
+from dataclasses import dataclass
+import warnings
+
+import numpy as np
+from scipy.optimize import root_scalar, minimize_scalar
+
+from . import utils
+
+
+def find_noise_multiplier_for_epsilon_delta(
+    accountant: "opacus.accountants.accountant.IAccountant",
+    sample_rate: float,
+    num_steps: int,
+    epsilon: float,
+    delta: float,
+    eps_error: float = 0.001,
+    mu_max: float = 100.0,
+    **accountant_kwargs,
+) -> float:
+    """
+    Find a noise multiplier that satisfies a given target epsilon.
+    Adapted from https://github.com/microsoft/prv_accountant/blob/main/prv_accountant/dpsgd.py
+
+    :param accountant: Opacus-compatible accountant
+    :param sample_rate: Probability of a record being in batch for Poisson sampling
+    :param num_steps: Number of optimisation steps
+    :param epsilon: Desired target epsilon
+    :param delta: Value of DP delta
+    :param float eps_error: Error allowed for final epsilon
+    :param float mu_max: Maximum value of noise multiplier of the search.
+    :param accountant_kwargs: Parameters passed to the accountant's `get_epsilon`
+    """
+
+    def compute_epsilon(mu: float) -> float:
+        acc = accountant()
+        for step in range(num_steps):
+            acc.step(noise_multiplier=mu, sample_rate=sample_rate)
+
+        return acc.get_epsilon(delta=delta, **accountant_kwargs)
+
+    mu_R = 1.0
+    eps_R = float("inf")
+    while eps_R > epsilon:
+        mu_R *= np.sqrt(2)
+        try:
+            eps_R = compute_epsilon(mu_R)
+        except (OverflowError, RuntimeError):
+            pass
+        if mu_R > mu_max:
+            raise RuntimeError(
+                "Finding a suitable noise multiplier has not converged. "
+                "Try increasing target epsilon or decreasing sample rate."
+            )
+
+    mu_L = mu_R
+    eps_L = eps_R
+    while eps_L <= epsilon:
+        mu_L /= np.sqrt(2)
+        eps_L = compute_epsilon(mu_L)
+
+    has_converged = False
+    bracket = [mu_L, mu_R]
+    while not has_converged:
+        mu_err = (bracket[1] - bracket[0]) * 0.01
+        assert mu_err > 0
+        mu_guess = root_scalar(
+            lambda mu: compute_epsilon(mu) - epsilon,
+            bracket=bracket,
+            xtol=mu_err,
+        ).root
+        bracket = [mu_guess - mu_err, mu_guess + mu_err]
+        eps_up = compute_epsilon(mu_guess - mu_err)
+        eps_low = compute_epsilon(mu_guess + mu_err)
+        has_converged = (eps_up - eps_low) < eps_error
+    assert compute_epsilon(bracket[1]) < epsilon + eps_error
+    return bracket[1]
+
+
+def find_noise_multiplier_for_advantage(
+    accountant: "opacus.accountants.accountant.IAccountant",
+    advantage: float,
+    sample_rate: float,
+    num_steps: float,
+    eps_error=0.001,
+    mu_max=100.0,
+    **accountant_kwargs,
+):
+    """
+    Find a noise multiplier that satisfies given levels of attack advantage.
+
+    :param accountant: Opacus-compatible accountant
+    :param advantage: Attack advantage bound
+    :param sample_rate: Probability of a record being in batch for Poisson sampling
+    :param num_steps: Number of optimisation steps
+    :param float eps_error: Error allowed for final epsilon
+    :param float mu_max: Maximum value of noise multiplier of the search
+    :param accountant_kwargs: Parameters passed to the accountant's `get_epsilon`
+    """
+    return find_noise_multiplier_for_epsilon_delta(
+        accountant=accountant,
+        sample_rate=sample_rate,
+        num_steps=num_steps,
+        epsilon=0.0,
+        delta=advantage,
+        eps_error=eps_error,
+        mu_max=mu_max,
+        **accountant_kwargs,
+    )
+
+
+class _ErrRatesAccountant:
+    def __init__(
+        self,
+        accountant,
+        alpha,
+        beta,
+        sample_rate,
+        num_steps,
+        eps_error,
+        mu_max=100.0,
+        **accountant_kwargs,
+    ):
+        self.accountant = accountant
+        self.alpha = alpha
+        self.beta = beta
+        self.sample_rate = sample_rate
+        self.num_steps = num_steps
+        self.eps_error = eps_error
+        self.mu_max = mu_max
+        self.accountant_kwargs = accountant_kwargs
+
+    def find_noise_multiplier(self, delta):
+        epsilon = utils.get_epsilon_for_err_rates(delta, self.alpha, self.beta)
+        try:
+            mu = find_noise_multiplier_for_epsilon_delta(
+                epsilon=epsilon,
+                delta=delta,
+                accountant=self.accountant,
+                sample_rate=self.sample_rate,
+                num_steps=self.num_steps,
+                eps_error=self.eps_error,
+                mu_max=self.mu_max,
+                **self.accountant_kwargs,
+            )
+            return mu
+
+        except RuntimeError as e:
+            warnings.warn(
+                f"Error occured in grid search w/ {epsilon=:.4f} {delta=:.4f}"
+            )
+            warnings.warn(e)
+            return np.inf
+
+
+def _find_noise_profile(
+    err_rates_acct_obj: _ErrRatesAccountant,
+    max_delta: float,
+    sample_rate: float,
+    num_steps: float,
+    delta_error=0.01,
+    eps_error=0.001,
+    mu_max=100.0,
+    **accountant_kwargs,
+):
+    delta_vals = np.linspace(
+        delta_error, max_delta, int((max_delta - delta_error) / delta_error)
+    )
+    if len(delta_vals) == 0:
+        raise ValueError("Grid resolution too low. Try increasing delta_error.")
+
+    noise_vals = np.array([np.inf] * len(delta_vals))
+    for i, delta in enumerate(delta_vals):
+        noise_vals[i] = err_rates_acct_obj.find_noise_multiplier(delta)
+
+    return delta_vals, noise_vals
+
+
+@dataclass
+class CalibrationResult:
+    """
+    Result of generic calibration.
+    """
+
+    noise_multiplier: float
+    calibration_epsilon: float
+    calibration_delta: float
+
+
+def find_noise_multiplier_for_err_rates(
+    accountant: "opacus.accountants.accountant.IAccountant",
+    alpha: float,
+    beta: float,
+    sample_rate: float,
+    num_steps: float,
+    delta_error=0.01,
+    eps_error=0.001,
+    mu_max=100.0,
+    method="brent",
+    **accountant_kwargs,
+):
+    """
+    Find a noise multiplier that limits attack FPR/FNR rates.
+
+    :param accountant: Opacus-compatible accountant
+    :param alpha: Attack FPR bound
+    :param beta: Attack FNR bound
+    :param sample_rate: Probability of a record being in batch for Poisson sampling
+    :param num_steps: Number of optimisation steps
+    :param float delta_error: Error allowed for delta used for calibration
+    :param float eps_error: Error allowed for final epsilon
+    :param float mu_max: Maximum value of noise multiplier of the search
+    :param str method: Optimization method. One of ['brent', 'grid_search']
+    :param accountant_kwargs: Parameters passed to the accountant's `get_epsilon`
+    """
+    if alpha + beta >= 1:
+        raise ValueError(
+            f"The guarantees are vacuous when alpha + beta >= 1. Got {alpha=}, {beta=}"
+        )
+
+    max_delta = 1 - alpha - beta
+    err_rates_acct_obj = _ErrRatesAccountant(
+        accountant=accountant,
+        alpha=alpha,
+        beta=beta,
+        sample_rate=sample_rate,
+        num_steps=num_steps,
+        eps_error=eps_error,
+        mu_max=mu_max,
+        **accountant_kwargs,
+    )
+
+    if max_delta < delta_error:
+        raise ValueError(f"{delta_error=} too low for the requested error rates.")
+
+    if method == "brent":
+        opt_result = minimize_scalar(
+            err_rates_acct_obj.find_noise_multiplier,
+            bounds=[delta_error, max_delta],
+            options=dict(xatol=delta_error),
+        )
+        if not opt_result.success:
+            raise RuntimeError(f"Optimization failed: {opt_result.message}")
+        calibration_delta = opt_result.x
+        noise_multiplier = opt_result.fun
+
+    elif method == "grid_search":
+        delta_vals, noise_vals = _find_noise_profile(
+            err_rates_acct_obj=err_rates_acct_obj,
+            max_delta=max_delta,
+            sample_rate=sample_rate,
+            num_steps=num_steps,
+            delta_error=delta_error,
+            eps_error=eps_error,
+            mu_max=mu_max,
+        )
+
+        noise_multiplier = noise_vals.min()
+        calibration_delta = delta_vals[np.argmin(noise_vals)]
+
+    return CalibrationResult(
+        noise_multiplier=noise_multiplier,
+        calibration_delta=calibration_delta,
+        calibration_epsilon=utils.get_epsilon_for_err_rates(
+            calibration_delta, alpha, beta
+        ),
+    )

riskcal-0.1.0/riskcal/pld.py

@@ -0,0 +1,317 @@
+from functools import reduce
+from typing import Union
+
+import numpy as np
+from dp_accounting.pld import privacy_loss_distribution
+from scipy.optimize import root_scalar
+
+
+class CTDAccountant:
+    """
+    Opacus-compatible Connect the Dots accountant.
+    """
+    def __init__(self):
+        self.history = []
+
+    def step(self, *, noise_multiplier, sample_rate):
+        if len(self.history) > 1:
+            prev_noise_multiplier, prev_sample_rate, prev_steps = self.history[-1]
+            if (
+                prev_noise_multiplier == noise_multiplier
+                and prev_sample_rate == sample_rate
+            ):
+                self.history[-1] = (noise_multiplier, sample_rate, prev_steps + 1)
+                return
+        self.history.append((noise_multiplier, sample_rate, 1))
+
+    def get_epsilon(self, *, delta, grid_step=1e-4, **kwargs):
+        plds = []
+        for noise_multiplier, sample_rate, num_steps in self.history:
+            pld = privacy_loss_distribution.from_gaussian_mechanism(
+                standard_deviation=noise_multiplier,
+                sampling_prob=sample_rate,
+                use_connect_dots=True,
+                value_discretization_interval=grid_step,
+            )
+            plds.append(pld.self_compose(num_steps))
+
+        composed_pld = reduce(lambda a, b: a.compose(b), plds)
+        return composed_pld.get_epsilon_for_delta(delta)
+
+    def __len__(self):
+        total = 0
+        for _, _, steps in self.history:
+            total += steps
+        return total
+
+    def mechanism(self):
+        return "ctd"
+
+    # The following methods are copied from https://opacus.ai/api/_modules/opacus/accountants/accountant.html#IAccountant
+    # to avoid the direct dependence on the opacus package.
+    def get_optimizer_hook_fn(
+        self, sample_rate: float
+    ):
+        """
+        Returns a callback function which can be used to attach to DPOptimizer
+        Args:
+            sample_rate: Expected sampling rate used for accounting
+        """
+
+        def hook_fn(optim):
+            # This works for Poisson for both single-node and distributed
+            # The reason is that the sample rate is the same in both cases (but in
+            # distributed mode, each node samples among a subset of the data)
+            self.step(
+                noise_multiplier=optim.noise_multiplier,
+                sample_rate=sample_rate * optim.accumulated_iterations,
+            )
+
+        return hook_fn
+
+    def state_dict(self, destination=None):
+        """
+        Returns a dictionary containing the state of the accountant.
+        Args:
+            destination: a mappable object to populate the current state_dict into.
+                If this arg is None, an OrderedDict is created and populated.
+                Default: None
+        """
+        if destination is None:
+            destination = OrderedDict()
+        destination["history"] = deepcopy(self.history)
+        destination["mechanism"] = self.__class__.mechanism
+        return destination
+
+    def load_state_dict(self, state_dict):
+        """
+        Validates the supplied state_dict and populates the current
+        Privacy Accountant's state dict.
+
+        Args:
+            state_dict: state_dict to load.
+
+        Raises:
+            ValueError if supplied state_dict is invalid and cannot be loaded.
+        """
+        if state_dict is None or len(state_dict) == 0:
+            raise ValueError(
+                "state dict is either None or empty and hence cannot be loaded"
+                " into Privacy Accountant."
+            )
+        if "history" not in state_dict.keys():
+            raise ValueError(
+                "state_dict does not have the key `history`."
+                " Cannot be loaded into Privacy Accountant."
+            )
+        if "mechanism" not in state_dict.keys():
+            raise ValueError(
+                "state_dict does not have the key `mechanism`."
+                " Cannot be loaded into Privacy Accountant."
+            )
+        if self.__class__.mechanism != state_dict["mechanism"]:
+            raise ValueError(
+                f"state_dict of {state_dict['mechanism']} cannot be loaded into "
+                f" Privacy Accountant with mechanism {self.__class__.mechanism}"
+            )
+        self.history = state_dict["history"]
+
+
+def _get_domain_and_pmf_from_pld(pld):
+    pld = pld.to_dense_pmf()
+    pmf = pld._probs
+    domain = (pld._lower_loss + np.arange(len(pld._probs))) * pld._discretization
+    return domain, pmf
+
+
+def _get_lower_loss_and_pmf_from_pld(pld):
+    pld = pld.to_dense_pmf()
+    pmf = pld._probs
+    lower_loss = pld._lower_loss
+    return lower_loss, pmf
+
+
+def get_beta_from_pld(
+    pld: privacy_loss_distribution.PrivacyLossDistribution,
+    alpha: Union[float, np.ndarray],
+):
+    lower_loss_Y, pmf_Y = _get_lower_loss_and_pmf_from_pld(
+        pld._pmf_remove
+    )
+    lower_loss_Z, pmf_Z = _get_lower_loss_and_pmf_from_pld(pld._pmf_add)
+
+    # Get the discrete points of alpha, beta
+    alphas = np.cumsum(pmf_Z) - pmf_Z
+    betas = np.cumsum(pmf_Y)
+
+    # Binary search to find the right index.
+    idx_Z = np.searchsorted(alphas, alpha) - 1
+
+    # Sanity check: did we find the correct index?
+    # Note that the alphas are in descending order.
+    assert np.all(alphas[idx_Z] < alpha) and np.all(alpha < alphas[idx_Z + 1])
+
+    # Find gamma.
+    gamma = (alpha - alphas[idx_Z]) / pmf_Z[idx_Z]
+
+    # Sanity check: gamma should be a positive and less than 1
+    assert np.all(0 < gamma) and np.all(gamma < 1)
+
+    # Get index in the Y world
+    idx_Y = -lower_loss_Z - lower_loss_Y - idx_Z
+
+    # Compute beta.
+    beta = betas[idx_Y] - gamma * pmf_Y[idx_Y]
+
+    # Sanity check: did we somehow go over to the next beta?
+    assert np.all(betas[idx_Y - 1] < beta) and np.all(beta < betas[idx_Y])
+
+    return beta
+
+
+def get_beta(
+    alpha: Union[float, np.ndarray],
+    noise_multiplier: float,
+    sample_rate: float,
+    num_steps: int,
+    grid_step=1e-4,
+):
+    """
+    Find FNR for a given FPR in DP-SGD.
+
+    Arguments:
+        alpha: Target FPR, either a single float or a numpy array
+        noise_multiplier: DP-SGD noise multiplier
+        sample_rate: Subsampled Gaussian sampling rate
+        num_steps: Number of steps
+        gird_step: Step size of the discretization grid
+    """
+    num_steps = int(num_steps)
+
+    pld = privacy_loss_distribution.from_gaussian_mechanism(
+        standard_deviation=noise_multiplier,
+        sampling_prob=sample_rate,
+        use_connect_dots=True,
+        value_discretization_interval=grid_step,
+    )
+    pld = pld.self_compose(num_steps)
+    return get_beta_from_pld(pld, alpha=alpha)
+
+
+def get_advantage(
+    noise_multiplier: float,
+    sample_rate: float,
+    num_steps: int,
+    grid_step=1e-4,
+):
+    google_pld = privacy_loss_distribution.from_gaussian_mechanism(
+        standard_deviation=noise_multiplier,
+        sampling_prob=sample_rate,
+        use_connect_dots=True,
+        value_discretization_interval=grid_step,
+    )
+    composed_google_pld = google_pld.self_compose(num_steps)
+    return composed_google_pld.get_delta_for_epsilon(0)
+
+
+def find_noise_multiplier_for_err_rates(
+    alpha: float,
+    beta: float,
+    sample_rate: float,
+    num_steps: float,
+    grid_step: float = 1e-4,
+    mu_max=100.0,
+    beta_error=0.001,
+):
+    """
+    Find a noise multiplier that satisfies a given target epsilon.
+    Adapted from https://github.com/microsoft/prv_accountant/blob/main/prv_accountant/dpsgd.py
+
+    :param alpha: Attack FPR bound
+    :param beta: Attack FNR bound
+    :param sample_rate: Probability of a record being in batch for Poisson sampling
+    :param num_steps: Number of optimisation steps
+    :param grid_step: Discretization grid step
+    :param delta: Value of DP delta
+    :param float mu_max: Maximum value of noise multiplier of the search.
+    """
+
+    def _get_beta(mu):
+        return get_beta(
+            noise_multiplier=mu,
+            alpha=alpha,
+            sample_rate=sample_rate,
+            num_steps=num_steps,
+            grid_step=grid_step,
+        )
+
+    mu_R = 1.0
+    beta_R = 0.0
+    while beta_R < beta:
+        mu_R *= np.sqrt(2)
+        try:
+            beta_R = _get_beta(mu_R)
+        except (OverflowError, RuntimeError):
+            pass
+        if mu_R > mu_max:
+            raise RuntimeError(
+                "Finding a suitable noise multiplier has not converged. "
+                "Try decreasing target beta or decreasing sample rate."
+            )
+
+    mu_L = mu_R
+    beta_L = beta_R
+    while beta_L > beta:
+        mu_L /= np.sqrt(2)
+        beta_L = _get_beta(mu_L)
+
+    has_converged = False
+    bracket = [mu_L, mu_R]
+    while not has_converged:
+        mu_err = (bracket[1] - bracket[0]) * 0.01
+
+        mu_guess = root_scalar(
+            lambda mu: _get_beta(mu) - beta,
+            bracket=bracket,
+            xtol=mu_err,
+        ).root
+        bracket = [mu_guess - mu_err, mu_guess + mu_err]
+        beta_low = _get_beta(mu_guess - mu_err)
+        beta_up = _get_beta(mu_guess + mu_err)
+        has_converged = (beta_up - beta_low) < beta_error
+
+    assert _get_beta(bracket[1]) > beta - beta_error
+    return bracket[1]
+
+
+def find_noise_multiplier_for_advantage(
+    advantage: float,
+    sample_rate: float,
+    num_steps: float,
+    grid_step: float = 1e-4,
+    mu_max=100.0,
+    advantage_error=0.001,
+):
+    """
+    Find a noise multiplier that satisfies a given target advantage.
+    Adapted from https://github.com/microsoft/prv_accountant/blob/main/prv_accountant/dpsgd.py
+
+    :param alpha: Attack FPR bound
+    :param beta: Attack FNR bound
+    :param sample_rate: Probability of a record being in batch for Poisson sampling
+    :param num_steps: Number of optimisation steps
+    :param grid_step: Discretization grid step
+    :param delta: Value of DP delta
+    :param float mu_max: Maximum value of noise multiplier of the search.
+    """
+    # Solve advantage = 1 - 2 * fp
+    fp = -0.5 * (advantage - 1)
+    return find_noise_multiplier_for_err_rates(
+        alpha=fp,
+        beta=fp,
+        sample_rate=sample_rate,
+        num_steps=num_steps,
+        grid_step=grid_step,
+        mu_max=mu_max,
+        beta_error=advantage_error,
+    )

riskcal-0.1.0/riskcal/utils.py

@@ -0,0 +1,66 @@
+import numpy as np
+from scipy.optimize import root_scalar
+
+
+def get_adv_for_epsilon_delta(epsilon: float, delta: float) -> float:
+    """Derive advantage from a given epsilon and delta.
+
+    >>> round(get_adv_for_epsilon_delta(0., 0.001), 3)
+    0.001
+    """
+    return (np.exp(epsilon) + 2 * delta - 1) / (np.exp(epsilon) + 1)
+
+
+def get_epsilon_for_advantage(delta: float, adv: float) -> float:
+    """Derive epsilon from a given advantage and delta.
+
+    >>> round(get_epsilon_for_advantage(0.001, 0.5), 3)
+    1.097
+    """
+    # We define the function to find the root of (f(eps) - 0) = 0
+    root_result = root_scalar(
+        lambda epsilon: get_adv_for_epsilon_delta(epsilon, delta) - adv,
+        method="brentq",
+        bracket=(0, 100),
+    )
+
+    if root_result.converged:
+        return root_result.root
+    else:
+        raise ValueError("Root finding did not converge.")
+
+
+def get_epsilon_for_err_rates(delta: float, alpha: float, beta: float):
+    """Derive epsilon from given FPR/FNR error rates and delta.
+
+    >>> round(get_epsilon_for_err_rates(0.001, 0.001, 0.8), 3)
+    5.293
+    """
+    epsilon1 = np.log((1 - delta - alpha) / beta)
+    epsilon2 = np.log((1 - delta - beta) / alpha)
+    return max(epsilon1, epsilon2, 0.0)
+
+
+def get_delta_for_err_rates(epsilon: float, alpha: float, beta: float):
+    """Derive delta from given FPR/FNR error rates and epsilon.
+
+    >>> round(get_delta_for_err_rates(1.0, 0.001, 0.8), 3)
+    0.197
+    """
+    delta1 = (1 - beta) - np.exp(epsilon) * alpha
+    delta2 = (1 - alpha) - np.exp(epsilon) * beta
+    return max(delta1, delta2, 0)
+
+
+def get_err_rate_for_epsilon_delta(epsilon: float, delta: float, alpha: float):
+    """Derive the error rate (e.g., FNR) for a given epsilon, delta, and the other error rate (e.g., FPR).
+
+    >>> round(get_err_rate_for_epsilon_delta(1.0, 0.001, 0.8), 3)
+    0.073
+    """
+    # See, e.g., Eq. 5 in https://arxiv.org/abs/1905.02383
+    return max(
+        0,
+        1 - delta - np.exp(epsilon) * alpha,
+        np.exp(-epsilon) * (1 - delta - alpha),
+    )