POPSRegression 0.1.0__tar.gz

popsregression-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 TD Swinburne (Tom)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

popsregression-0.1.0/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: POPSRegression
+Version: 0.1.0
+Summary: Bayesian regression for low-noise data using POPS algorithm
+Author-email: Thomas D Swinburne <thomas.swinburne@cnrs.fr>, Danny Perez <danny_perez@lanl.gov>
+License: MIT
+Project-URL: Homepage, https://github.com/tomswinburne/POPS-Regression
+Project-URL: Bug Tracker, https://github.com/tomswinburne/POPS-Regression/issues
+Project-URL: Documentation, https://github.com/tomswinburne/POPS-Regression/blob/main/README.md
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: scikit-learn>=0.24.0
+Requires-Dist: scipy>=1.6.0
+Requires-Dist: numpy>=1.20.0
+
+# POPSRegression
+Regression scheme from the paper 
+
+*Parameter uncertainties for imperfect surrogate models in the low-noise regime*
+
+TD Swinburne and D Perez, [arXiv 2024](https://arxiv.org/abs/2402.01810v3)
+
+```bibtex
+@misc{swinburne2024,
+      title={Parameter uncertainties for imperfect surrogate models in the low-noise regime}, 
+      author={Thomas D Swinburne and Danny Perez},
+      year={2024},
+      eprint={2402.01810},
+      archivePrefix={arXiv},
+      primaryClass={stat.ML},
+      url={https://arxiv.org/abs/2402.01810v3}, 
+}
+```
+
+
+Bayesian regression for low-noise data (vanishing aleatoric uncertainty). 
+
+Fits the weights of a regression model using BayesianRidge, then estimates weight uncertainties accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm. 
+
+The method can easily handle high-dimensional linear problems with minimal overhead compared to any linear regression scheme.
+
+Bayesian regression is often used in computational science to fit the weights of a surrogate model which approximates some complex calcualtion. 
+In many important cases the target calcualtion is near-deterministic, or low-noise, meaning the true data has vanishing aleatoric uncertainty. 
+However, there can be large misspecification uncertainty, i.e. the model weights are instrinsically uncertain as the model is unable to exactly match training data. 
+Existing Bayesian regression schemes based on loss minimization can only estimate epistemic and aleatoric uncertainties. 
+
+## Example usage
+Here, usage follows `sklearn.linear_model`, inheriting `BayesianRidge`
+
+After running `BayesianRidge.fit(..)`, the `alpha_` attribute is fixed to `np.inf` as aleatoric uncertainty is assumed negligable.
+
+The `sigma_` matrix still contains epistemic weight uncertainties, whilst `misspecification_sigma_` contains the POPS uncertainties. 
+
+```python
+
+from POPSRegression import POPSRegression
+
+X_train,X_test,y_train,y_test = ...
+
+# Sobol resampling of hypercube with 1.0 samples / training point
+model = POPSRegression(resampling_method='sobol',resample_density=1.)
+
+# fit the model, sample POPS hypercube
+model.fit(X_train,y_train)
+
+# Return hypercube std, max/min and epistemic uncertaint from inference
+y_pred, y_std, y_max, y_min, y_std_epistmic = \
+    model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+
+
+
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+
+# returns std by default
+y_pred, y_std = model.predict(X_test)
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+# can also resample the hypercube vectors
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True,resample=True)
+
+# can also return the epistemic uncertainty (descreases as 1/sqrt(n_samples))
+y_pred, y_std, y_max, y_min, y_std_epistmic = model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+```
+
+As can be seen, the final error bars give very good coverage of the test output
+
+Extreme low-dimensional case, fitting N data points to a quartic polynomial (P=5 parameters) to some complex oscillatory function
+
+Green: two sigma of `sigma_` weight uncertainty from Bayesian Regression (i.e. without `alpha_` term for aleatoric error)
+
+Orange: two sigma of `sigma_` and `misspecification_sigma_` posterior from POPS Regression
+
+![Example POPS regression](example_image.png)
+

popsregression-0.1.0/POPSRegression/POPSRegression.py

@@ -0,0 +1,394 @@
+import numpy as np
+from sklearn.linear_model import BayesianRidge
+from scipy.linalg import pinvh, eigh
+from sklearn.base import BaseEstimator, RegressorMixin, _fit_context
+from sklearn.utils._param_validation import Interval, StrOptions
+from numbers import Real, Integral
+from scipy.stats import qmc 
+
+
+###############################################################################
+# POPS (Pointwise Optimal Parameter Sets) regression
+
+
+class POPSRegression(BayesianRidge):
+    """
+    Bayesian regression for low-noise data (vanishing aleatoric uncertainty). 
+
+    Fits the weights of a regression model using BayesianRidge, then estimates weight uncertainties (`sigma_` in `BayesianRidge`) accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm [1]. The default`alpha_` attribute is fixed to `np.inf` as aleatoric uncertainty is assumed negligable.
+
+    Bayesian regression is often used in computational science to fit the weights of a surrogate model which approximates some complex calcualtion. 
+    In many important cases the target calcualtion is near-deterministic, or low-noise, meaning the true data has vanishing aleatoric uncertainty. However, there can be large misspecification uncertainty, i.e. the model weights are instrinsically uncertain as the model is unable to exactly match training data. 
+
+    Existing Bayesian regression schemes based on loss minimization can only estimate epistemic and aleatoric uncertainties. In the low-noise limit, 
+    weight uncertainties (`sigma_` in `BayesianRidge`) are significantly underestimated as they only account for epistemic uncertainties which decay with increasing data. Predictions then assume any additional error is due to an aleatoric uncertainty (`alpha_` in `BayesianRidge`), which is erroneous in a low-noise setting. This has significant implications on how uncertainty is propagated using weight uncertainties. 
+
+    Parameters
+    ----------
+    max_iter : int, default=300
+        Maximum number of iterations.
+    tol : float, default=1e-3
+        Stop the algorithm if w has converged.
+    alpha_1 : float, default=1e-6
+        Hyper-parameter : shape parameter for the Gamma distribution prior over the alpha parameter.
+    alpha_2 : float, default=1e-6
+        Hyper-parameter : inverse scale parameter (rate parameter) for the Gamma distribution prior over the alpha parameter.
+    lambda_1 : float, default=1e-6
+        Hyper-parameter : shape parameter for the Gamma distribution prior over the lambda parameter.
+    lambda_2 : float, default=1e-6
+        Hyper-parameter : inverse scale parameter (rate parameter) for the Gamma distribution prior over the lambda parameter.
+    alpha_init : float, default=None
+        Initial value for alpha (precision of the noise).
+    lambda_init : float, default=None
+        Initial value for lambda (precision of the weights).
+    compute_score : bool, default=False
+        If True, compute the objective function at each step of the model.
+    fit_intercept : bool, default=False
+        Whether to calculate the intercept for this model.
+    copy_X : bool, default=True
+        If True, X will be copied; else, it may be overwritten.
+    verbose : bool, default=False
+        Verbose mode when fitting the model.
+    mode_threshold : float, default=1e-3
+        Threshold for determining the mode of the posterior distribution.
+    resample_density : float, default=1.0
+        Density of resampling for the POPS algorithm- number of hypercube per training point. Default is the greater of 0.5 or 100/n_samples.
+    resampling_method : str, default='uniform'
+        Method of resampling for the POPS algorithm. 
+        must be one of 'sobol', 'latin', 'halton', 'grid', or 'uniform'.
+    percentile_clipping : float, default=0.0
+        Percentile to clip from each end of the distribution when determining the hypercube bounds, i.e. spans [x,100-x]. Must be between 0 and 50, but in practice should be between 0.0% and 0.5% for robust bounds
+    Attributes
+    ----------
+    coef_ : array-like of shape (n_features,)
+        Coefficients of the regression model (mean of distribution).
+    intercept_ : float
+        Independent term in decision function. Set to 0.0 if
+        `fit_intercept = False`.
+    alpha_ : float
+        Estimated precision of the noise.
+    lambda_ : float
+        Estimated precision of the weights.
+    sigma_ : array-like of shape (n_features, n_features)
+        Estimated variance-covariance matrix of the weights.
+    scores_ : list
+        If computed, value of the objective function (to be maximized).
+
+    Notes
+    -----
+    The POPS algorithm extends Bayesian Ridge Regression by incorporating
+    probabilistic optimization of predictive subspaces, which can lead to
+    improved performance in high-dimensional settings.
+
+    References
+    ----------
+    .. [1] Swinburne, T.D and Perez, D (2024). 
+           Parameter uncertainties for imperfect surrogate models in the low-noise regime, arXiv:2402.01810v3
+    """
+    _parameter_constraints: dict = {
+        "max_iter": [Interval(Integral, 1, None, closed="left")],
+        "tol": [Interval(Real, 0, None, closed="neither")],
+        "alpha_1": [Interval(Real, 0, None, closed="left")],
+        "alpha_2": [Interval(Real, 0, None, closed="left")],
+        "lambda_1": [Interval(Real, 0, None, closed="left")],
+        "lambda_2": [Interval(Real, 0, None, closed="left")],
+        "alpha_init": [None, Interval(Real, 0, None, closed="left")],
+        "lambda_init": [None, Interval(Real, 0, None, closed="left")],
+        "compute_score": ["boolean"],
+        "fit_intercept": ["boolean"],
+        "copy_X": ["boolean"],
+        "verbose": ["verbose"],
+        "resampling_method": [StrOptions({"uniform","sobol","latin","halton"})],
+        "mode_threshold": [Interval(Real, 0, None, closed="neither")],
+        "resample_density": [Interval(Real, 0, None, closed="neither")],
+        "percentile_clipping": [Interval(Real, 0, 50., closed="both")]
+    }
+    def __init__(
+        self,
+        *,
+        max_iter=300,
+        tol=1.0e-3,
+        alpha_1=1.0e-6,
+        alpha_2=1.0e-6,
+        lambda_1=1.0e-6,
+        lambda_2=1.0e-6,
+        alpha_init=None,
+        lambda_init=None,
+        compute_score=False,
+        fit_intercept=False,
+        copy_X=True,
+        verbose=False,
+        mode_threshold=1.0e-8,
+        resample_density=1.0,
+        resampling_method='uniform',
+        percentile_clipping=0.0,
+    ):
+        super().__init__(
+            max_iter=max_iter,
+            tol=tol,
+            alpha_1=alpha_1,
+            alpha_2=alpha_2,
+            lambda_1=lambda_1,
+            lambda_2=lambda_2,
+            alpha_init=alpha_init,
+            lambda_init=lambda_init,
+            compute_score=compute_score,
+            fit_intercept=fit_intercept,
+            copy_X=copy_X,
+            verbose=verbose,
+        )
+        self.mode_threshold = mode_threshold
+        self.fit_intercept_flag = False
+        self.resample_density = resample_density
+        self.resampling_method = resampling_method
+        self.percentile_clipping = percentile_clipping
+        self._validate_params()
+
+        if self.fit_intercept:
+            print("Warning: fit_intercept is set to False for POPS regression. A constant feature will be added to the design matrix.")
+            self.fit_intercept_flag = True
+            self.fit_intercept = False
+    
+    @_fit_context(prefer_skip_nested_validation=True)
+    def fit(self, X, y, sample_weight=None):
+        """
+        Fit the POPS regression model.
+
+        This method extends the fit method of BayesianRidge to include POPS-specific
+        computations, such as calculating leverage scores, pointwise corrections,
+        and determining the hypercube support.
+
+        Parameters:
+        -----------
+        X : array-like of shape (n_samples, n_features)
+            The input samples.
+        y : array-like of shape (n_samples,)
+            The target values.
+        sample_weight : array-like of shape (n_samples,), default=None
+            Individual weights for each sample.
+
+        Returns:
+        --------
+        self : object
+            Returns the instance itself.
+        """
+        if self.fit_intercept_flag:
+            print("Warning: fit_intercept is set to False for POPS regression. Adding a constant feature for regression.")
+            X = np.hstack([X, np.ones((X.shape[0], 1))])
+        
+        super().fit(X, y, sample_weight)
+        
+        # suppress aleatoric uncertainty
+        self.alpha_ = np.inf 
+
+        n_features = X.shape[1]
+        n_samples = X.shape[0]
+
+        # Prior is lambda_ from BayesianRidge
+        prior = self.lambda_ * np.eye(n_features) / n_samples
+        
+        # Prior ensures that the design matrix is invertible
+        inverse_design_matrix = pinvh(X.T @ X / n_samples + prior)
+
+        # Calculate leverage and pointwise fits
+        errors = y - X @ self.coef_  # errors to mean prediction
+        X_inverse_design_matrix = X @ inverse_design_matrix
+        
+        self.leverage_scores = (X_inverse_design_matrix * X).sum(1)
+        
+        self.pointwise_correction = X_inverse_design_matrix \
+            * (errors/self.leverage_scores)[:,None]
+        
+        # Determine bounding hypercube from pointwise fits
+        self.hypercube_support, self.hypercube_bounds = \
+            self._hypercube_fit(self.pointwise_correction,\
+                                self.percentile_clipping)
+        
+        self.hypercube_samples,self.misspecification_sigma_ = \
+            self._resample_hypercube()
+
+    def _hypercube_fit(self,pointwise_correction,percentile_clipping=0.0):
+        """
+        Fit a hypercube to the pointwise corrections.
+
+        This method calculates the principal components of the pointwise corrections
+        and determines the bounding box (hypercube) in the space of these components.
+
+        Parameters:
+        -----------
+        pointwise_correction : numpy.ndarray
+            Array of pointwise corrections, shape (n_samples, n_features).
+
+        Returns:
+        --------
+        projections : numpy.ndarray
+            The principal component vectors that define the hypercube space.
+        bounds : numpy.ndarray
+            The min and max bounds of the hypercube along each principal component.
+
+        Notes:
+        ------
+        The method performs the following steps:
+        1. Compute the eigendecomposition of the covariance matrix of pointwise corrections.
+        2. Select principal components based on the mode_threshold.
+        3. Project the pointwise corrections onto these components.
+        4. Determine the bounding box (hypercube) in this projected space.
+
+        The resulting hypercube represents the uncertainty in the parameter estimates,
+        which can be used for subsequent resampling and uncertainty quantification.
+        """
+        
+        e_values, e_vectors = eigh(pointwise_correction.T @pointwise_correction)
+        
+        mask = e_values > self.mode_threshold * e_values.max()
+        e_vectors = e_vectors[:,mask]
+        e_values = e_values[mask]
+        
+        projections = e_vectors.copy()
+        projected = pointwise_correction @ projections
+        bounds = [np.percentile(projected,percentile_clipping,axis=0)]
+        bounds += [np.percentile(projected,100.-percentile_clipping,axis=0)]
+        
+        return projections, bounds
+    
+    def _resample_hypercube(self,size=None,resampling_method=None):
+        """
+        Resample points from the hypercube.
+
+        This method generates new samples from the hypercube defined by the
+        bounding box of the pointwise corrections. The sampling is uniform
+        within the hypercube.
+
+        Parameters:
+        -----------
+        size : int, optional
+            The number of samples to generate. If None, the number of samples
+            is determined by self.resample_density * self.leverage_scores.size.
+
+        Returns:
+        --------
+        numpy.ndarray
+            An array of shape (n_features, n_samples) containing the resampled
+            points in the feature space.
+
+        Notes:
+        ------
+        The resampling process involves the following steps:
+        1. Generate uniform random numbers between 0 and 1.
+        2. Scale these numbers to the range of the hypercube bounds.
+        3. Project the scaled points back to the original feature space using
+           the hypercube support vectors.
+
+        This method is used to generate new possible parameter values within
+        the uncertainty bounds of the model, which can be used for uncertainty
+        quantification in predictions.
+        """
+        if resampling_method is None:
+            resampling_method = self.resampling_method
+        
+        # Validate resampling_method parameter
+        valid_methods = ['latin', 'sobol', 'grid', 'halton', 'uniform']
+        if resampling_method not in valid_methods:
+            raise ValueError(f"Invalid resampling_method. Must be one of {valid_methods}")
+
+        low = self.hypercube_bounds[0]
+        high = self.hypercube_bounds[1]
+        if size is None:
+            n_resample = int(self.resample_density*self.leverage_scores.size)
+        else:
+            n_resample = size
+        n_resample = max(n_resample,100)
+        
+        # Sobol sequence
+        if resampling_method == 'latin':
+            sampler = qmc.LatinHypercube(d=low.size)
+            samples = sampler.random(n_resample).T 
+        elif resampling_method == 'sobol':
+            sampler = qmc.Sobol(d=low.size)
+            n_resample = 2**int(np.log(n_resample)/np.log(2.0))
+            samples = sampler.random(n_resample).T 
+        elif resampling_method == 'grid':
+            samples = np.linspace(0,1,n_resample).T
+        elif resampling_method == 'halton':
+            sampler = qmc.Halton(d=low.size)
+            samples = sampler.random(n_resample).T 
+        elif resampling_method == 'uniform':
+            samples = np.random.uniform(size=(low.size,n_resample))
+        samples = low[:,None] + (high-low)[:,None]*samples
+
+        hypercube_samples = self.hypercube_support@samples
+        hypercube_sigma = hypercube_samples@hypercube_samples.T
+        hypercube_sigma /= hypercube_samples.shape[1]
+
+        return hypercube_samples,hypercube_sigma
+    def resample(self,resampling_method=None):
+        """
+        Resample the hypercube samples and update the model's internal state.
+
+        This method calls the _resample_hypercube method to generate new samples
+        and update the hypercube_samples and hypercube_sigma attributes of the model.
+
+        Parameters:
+        -----------
+        resampling_method : str, optional (default=None)
+            The method to use for resampling. If None, the model's default
+            resampling_method will be used. Valid options are:
+            'latin', 'sobol', 'grid', 'halton', 'uniform'.
+
+        Returns:
+        --------
+        None
+
+        Notes:
+        ------
+        This method is used to update the model's uncertainty estimates by
+        generating new samples within the hypercube bounds. It's particularly
+        useful for uncertainty quantification in predictions and can be called
+        multiple times to get different uncertainty estimates.
+        """
+        self._resample_hypercube(resampling_method=resampling_method)
+
+    def predict(self,X,return_bounds=False,return_epistemic_std=False):
+        """
+        Make predictions using the POPS model.
+
+        Parameters:
+        -----------
+        X : array-like of shape (n_samples, n_features)
+            The input samples for prediction.
+        return_bounds : bool, default=False
+            If True, return the min and max bounds of the prediction.
+        return_epistemic_std : bool, default=False
+            If True, return the epistemic standard deviation.
+
+        Returns:
+        --------
+        y_pred : array-like of shape (n_samples,)
+            The predicted mean values.
+        y_std : array-like of shape (n_samples,)
+            The predicted standard deviation (uncertainty) for each prediction.
+        y_max : array-like of shape (n_samples,), optional
+            The upper bound of the prediction interval. Only returned if return_bounds is True.
+        y_min : array-like of shape (n_samples,), optional
+            The lower bound of the prediction interval. Only returned if return_bounds is True.
+        """
+        
+        # DeterministicBayesianRidge suppresses aleatoric uncertainty
+        y_pred, y_epistemic_std = \
+            super().predict(X,return_std=True)
+        
+        
+        # Combine misspecification and epistemic uncertainty
+        y_misspecification_var = (X@self.misspecification_sigma_ * X).sum(1)
+        y_std = np.sqrt(y_misspecification_var + y_epistemic_std**2)
+        
+        res = [y_pred, y_std]
+        if return_bounds:
+            y_max = (X@self.hypercube_samples).max(1) + y_pred
+            y_min = (X@self.hypercube_samples).min(1) + y_pred
+            res += [y_max, y_min]
+        
+        if return_epistemic_std:
+            res += [y_epistemic_std]
+        
+        return tuple(res)

popsregression-0.1.0/POPSRegression/__init__.py

@@ -0,0 +1 @@
+from .POPSRegression import POPSRegression

popsregression-0.1.0/POPSRegression.egg-info/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: POPSRegression
+Version: 0.1.0
+Summary: Bayesian regression for low-noise data using POPS algorithm
+Author-email: Thomas D Swinburne <thomas.swinburne@cnrs.fr>, Danny Perez <danny_perez@lanl.gov>
+License: MIT
+Project-URL: Homepage, https://github.com/tomswinburne/POPS-Regression
+Project-URL: Bug Tracker, https://github.com/tomswinburne/POPS-Regression/issues
+Project-URL: Documentation, https://github.com/tomswinburne/POPS-Regression/blob/main/README.md
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: scikit-learn>=0.24.0
+Requires-Dist: scipy>=1.6.0
+Requires-Dist: numpy>=1.20.0
+
+# POPSRegression
+Regression scheme from the paper 
+
+*Parameter uncertainties for imperfect surrogate models in the low-noise regime*
+
+TD Swinburne and D Perez, [arXiv 2024](https://arxiv.org/abs/2402.01810v3)
+
+```bibtex
+@misc{swinburne2024,
+      title={Parameter uncertainties for imperfect surrogate models in the low-noise regime}, 
+      author={Thomas D Swinburne and Danny Perez},
+      year={2024},
+      eprint={2402.01810},
+      archivePrefix={arXiv},
+      primaryClass={stat.ML},
+      url={https://arxiv.org/abs/2402.01810v3}, 
+}
+```
+
+
+Bayesian regression for low-noise data (vanishing aleatoric uncertainty). 
+
+Fits the weights of a regression model using BayesianRidge, then estimates weight uncertainties accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm. 
+
+The method can easily handle high-dimensional linear problems with minimal overhead compared to any linear regression scheme.
+
+Bayesian regression is often used in computational science to fit the weights of a surrogate model which approximates some complex calcualtion. 
+In many important cases the target calcualtion is near-deterministic, or low-noise, meaning the true data has vanishing aleatoric uncertainty. 
+However, there can be large misspecification uncertainty, i.e. the model weights are instrinsically uncertain as the model is unable to exactly match training data. 
+Existing Bayesian regression schemes based on loss minimization can only estimate epistemic and aleatoric uncertainties. 
+
+## Example usage
+Here, usage follows `sklearn.linear_model`, inheriting `BayesianRidge`
+
+After running `BayesianRidge.fit(..)`, the `alpha_` attribute is fixed to `np.inf` as aleatoric uncertainty is assumed negligable.
+
+The `sigma_` matrix still contains epistemic weight uncertainties, whilst `misspecification_sigma_` contains the POPS uncertainties. 
+
+```python
+
+from POPSRegression import POPSRegression
+
+X_train,X_test,y_train,y_test = ...
+
+# Sobol resampling of hypercube with 1.0 samples / training point
+model = POPSRegression(resampling_method='sobol',resample_density=1.)
+
+# fit the model, sample POPS hypercube
+model.fit(X_train,y_train)
+
+# Return hypercube std, max/min and epistemic uncertaint from inference
+y_pred, y_std, y_max, y_min, y_std_epistmic = \
+    model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+
+
+
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+
+# returns std by default
+y_pred, y_std = model.predict(X_test)
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+# can also resample the hypercube vectors
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True,resample=True)
+
+# can also return the epistemic uncertainty (descreases as 1/sqrt(n_samples))
+y_pred, y_std, y_max, y_min, y_std_epistmic = model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+```
+
+As can be seen, the final error bars give very good coverage of the test output
+
+Extreme low-dimensional case, fitting N data points to a quartic polynomial (P=5 parameters) to some complex oscillatory function
+
+Green: two sigma of `sigma_` weight uncertainty from Bayesian Regression (i.e. without `alpha_` term for aleatoric error)
+
+Orange: two sigma of `sigma_` and `misspecification_sigma_` posterior from POPS Regression
+
+![Example POPS regression](example_image.png)
+

popsregression-0.1.0/POPSRegression.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+LICENSE
+README.md
+pyproject.toml
+POPSRegression/POPSRegression.py
+POPSRegression/__init__.py
+POPSRegression.egg-info/PKG-INFO
+POPSRegression.egg-info/SOURCES.txt
+POPSRegression.egg-info/dependency_links.txt
+POPSRegression.egg-info/requires.txt
+POPSRegression.egg-info/top_level.txt

popsregression-0.1.0/POPSRegression.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

popsregression-0.1.0/POPSRegression.egg-info/requires.txt

@@ -0,0 +1,3 @@
+scikit-learn>=0.24.0
+scipy>=1.6.0
+numpy>=1.20.0

popsregression-0.1.0/POPSRegression.egg-info/top_level.txt

@@ -0,0 +1 @@
+POPSRegression

popsregression-0.1.0/README.md

@@ -0,0 +1,84 @@
+# POPSRegression
+Regression scheme from the paper 
+
+*Parameter uncertainties for imperfect surrogate models in the low-noise regime*
+
+TD Swinburne and D Perez, [arXiv 2024](https://arxiv.org/abs/2402.01810v3)
+
+```bibtex
+@misc{swinburne2024,
+      title={Parameter uncertainties for imperfect surrogate models in the low-noise regime}, 
+      author={Thomas D Swinburne and Danny Perez},
+      year={2024},
+      eprint={2402.01810},
+      archivePrefix={arXiv},
+      primaryClass={stat.ML},
+      url={https://arxiv.org/abs/2402.01810v3}, 
+}
+```
+
+
+Bayesian regression for low-noise data (vanishing aleatoric uncertainty). 
+
+Fits the weights of a regression model using BayesianRidge, then estimates weight uncertainties accounting for model misspecification using the POPS (Pointwise Optimal Parameter Sets) algorithm. 
+
+The method can easily handle high-dimensional linear problems with minimal overhead compared to any linear regression scheme.
+
+Bayesian regression is often used in computational science to fit the weights of a surrogate model which approximates some complex calcualtion. 
+In many important cases the target calcualtion is near-deterministic, or low-noise, meaning the true data has vanishing aleatoric uncertainty. 
+However, there can be large misspecification uncertainty, i.e. the model weights are instrinsically uncertain as the model is unable to exactly match training data. 
+Existing Bayesian regression schemes based on loss minimization can only estimate epistemic and aleatoric uncertainties. 
+
+## Example usage
+Here, usage follows `sklearn.linear_model`, inheriting `BayesianRidge`
+
+After running `BayesianRidge.fit(..)`, the `alpha_` attribute is fixed to `np.inf` as aleatoric uncertainty is assumed negligable.
+
+The `sigma_` matrix still contains epistemic weight uncertainties, whilst `misspecification_sigma_` contains the POPS uncertainties. 
+
+```python
+
+from POPSRegression import POPSRegression
+
+X_train,X_test,y_train,y_test = ...
+
+# Sobol resampling of hypercube with 1.0 samples / training point
+model = POPSRegression(resampling_method='sobol',resample_density=1.)
+
+# fit the model, sample POPS hypercube
+model.fit(X_train,y_train)
+
+# Return hypercube std, max/min and epistemic uncertaint from inference
+y_pred, y_std, y_max, y_min, y_std_epistmic = \
+    model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+
+
+
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+
+# returns std by default
+y_pred, y_std = model.predict(X_test)
+
+# can also return max/min 
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True)
+
+# can also resample the hypercube vectors
+y_pred, y_std, y_max, y_min = model.predict(X_test,return_bounds=True,resample=True)
+
+# can also return the epistemic uncertainty (descreases as 1/sqrt(n_samples))
+y_pred, y_std, y_max, y_min, y_std_epistmic = model.predict(X_test,return_bounds=True,resample=True,return_epistemic_std=True)
+```
+
+As can be seen, the final error bars give very good coverage of the test output
+
+Extreme low-dimensional case, fitting N data points to a quartic polynomial (P=5 parameters) to some complex oscillatory function
+
+Green: two sigma of `sigma_` weight uncertainty from Bayesian Regression (i.e. without `alpha_` term for aleatoric error)
+
+Orange: two sigma of `sigma_` and `misspecification_sigma_` posterior from POPS Regression
+
+![Example POPS regression](example_image.png)
+

popsregression-0.1.0/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=45", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "POPSRegression"
+version = "0.1.0"
+description = "Bayesian regression for low-noise data using POPS algorithm"
+authors = [
+    {name = "Thomas D Swinburne", email = "thomas.swinburne@cnrs.fr"},
+    {name = "Danny Perez", email = "danny_perez@lanl.gov"},
+]
+license = {text = "MIT"}
+readme = "README.md"
+requires-python = ">=3.10"
+
+dependencies = [
+    "scikit-learn>=0.24.0",
+    "scipy>=1.6.0",
+    "numpy>=1.20.0",
+]
+[project.urls]
+"Homepage" = "https://github.com/tomswinburne/POPS-Regression"
+"Bug Tracker" = "https://github.com/tomswinburne/POPS-Regression/issues"
+"Documentation" = "https://github.com/tomswinburne/POPS-Regression/blob/main/README.md"
+
+[tool.setuptools]
+packages = ["POPSRegression"]
+

popsregression-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+