spotoptim 0.0.3__tar.gz → 0.0.5__tar.gz

spotoptim-0.0.5/PKG-INFO

@@ -0,0 +1,150 @@
+Metadata-Version: 2.3
+Name: spotoptim
+Version: 0.0.5
+Summary: Add your description here
+Author: bartzbeielstein
+Author-email: bartzbeielstein <32470350+bartzbeielstein@users.noreply.github.com>
+Requires-Dist: numpy>=1.24.3
+Requires-Dist: scipy>=1.10.1
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+
+# SpotOptim
+
+Sequential Parameter Optimization with Bayesian Optimization.
+
+## Features
+
+- **Bayesian Optimization**: Uses surrogate models to efficiently optimize expensive black-box functions
+- **Multiple Acquisition Functions**: Expected Improvement (EI), Predicted Mean (y), Probability of Improvement (PI)
+- **Flexible Surrogates**: Default Gaussian Process or custom Kriging surrogate
+- **Variable Types**: Support for continuous, integer, and mixed variable types
+- **scipy-compatible**: Returns OptimizeResult objects compatible with scipy.optimize
+
+## Installation
+
+```bash
+pip install spotoptim
+```
+
+## Quick Start
+
+```python
+import numpy as np
+from spotoptim import SpotOptim
+
+# Define objective function
+def rosenbrock(X):
+    X = np.atleast_2d(X)
+    x, y = X[:, 0], X[:, 1]
+    return (1 - x)**2 + 100 * (y - x**2)**2
+
+# Set up optimization
+bounds = [(-2, 2), (-2, 2)]
+
+optimizer = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    max_iter=50,
+    n_initial=10,
+    seed=42
+)
+
+# Run optimization
+result = optimizer.optimize()
+
+print(f"Best point: {result.x}")
+print(f"Best value: {result.fun}")
+```
+
+## Using Kriging Surrogate
+
+SpotOptim includes a simplified Kriging (Gaussian Process) surrogate as an alternative to scikit-learn's GaussianProcessRegressor:
+
+```python
+from spotoptim import SpotOptim, Kriging
+
+# Create Kriging surrogate
+kriging = Kriging(
+    noise=1e-6,
+    min_theta=-3.0,
+    max_theta=2.0,
+    seed=42
+)
+
+# Use with SpotOptim
+optimizer = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    surrogate=kriging,  # Use Kriging instead of default GP
+    seed=42
+)
+
+result = optimizer.optimize()
+```
+
+## API Reference
+
+### SpotOptim
+
+**Parameters:**
+- `fun` (callable): Objective function to minimize
+- `bounds` (list of tuples): Bounds for each dimension as [(low, high), ...]
+- `max_iter` (int, default=20): Maximum number of optimization iterations
+- `n_initial` (int, default=10): Number of initial design points
+- `surrogate` (object, optional): Surrogate model (default: GaussianProcessRegressor)
+- `acquisition` (str, default='ei'): Acquisition function ('ei', 'y', 'pi')
+- `var_type` (list of str, optional): Variable types for each dimension
+- `tolerance_x` (float, optional): Minimum distance between points
+- `seed` (int, optional): Random seed for reproducibility
+- `verbose` (bool, default=False): Print progress information
+
+**Methods:**
+- `optimize(X0=None)`: Run optimization, optionally with initial design points
+
+### Kriging
+
+**Parameters:**
+- `noise` (float, optional): Regularization parameter
+- `kernel` (str, default='gauss'): Kernel type
+- `n_theta` (int, optional): Number of theta parameters
+- `min_theta` (float, default=-3.0): Minimum log10(theta) bound
+- `max_theta` (float, default=2.0): Maximum log10(theta) bound
+- `seed` (int, optional): Random seed
+
+**Methods:**
+- `fit(X, y)`: Fit the model to training data
+- `predict(X, return_std=False)`: Predict at new points
+
+## Examples
+
+See the `notebooks/demos.ipynb` for comprehensive examples including:
+1. 2D Rosenbrock function optimization
+2. 6D Rosenbrock with budget constraints
+3. Using Kriging surrogate vs default GP
+
+## Development
+
+```bash
+# Clone repository
+git clone https://github.com/sequential-parameter-optimization/spotoptim.git
+cd spotoptim
+
+# Install with uv
+uv pip install -e .
+
+# Run tests
+uv run pytest tests/
+
+# Build package
+uv build
+```
+
+## License
+
+See LICENSE file.
+
+## References
+
+Based on the SPOT (Sequential Parameter Optimization Toolbox) methodology.

spotoptim-0.0.5/README.md

@@ -0,0 +1,138 @@
+# SpotOptim
+
+Sequential Parameter Optimization with Bayesian Optimization.
+
+## Features
+
+- **Bayesian Optimization**: Uses surrogate models to efficiently optimize expensive black-box functions
+- **Multiple Acquisition Functions**: Expected Improvement (EI), Predicted Mean (y), Probability of Improvement (PI)
+- **Flexible Surrogates**: Default Gaussian Process or custom Kriging surrogate
+- **Variable Types**: Support for continuous, integer, and mixed variable types
+- **scipy-compatible**: Returns OptimizeResult objects compatible with scipy.optimize
+
+## Installation
+
+```bash
+pip install spotoptim
+```
+
+## Quick Start
+
+```python
+import numpy as np
+from spotoptim import SpotOptim
+
+# Define objective function
+def rosenbrock(X):
+    X = np.atleast_2d(X)
+    x, y = X[:, 0], X[:, 1]
+    return (1 - x)**2 + 100 * (y - x**2)**2
+
+# Set up optimization
+bounds = [(-2, 2), (-2, 2)]
+
+optimizer = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    max_iter=50,
+    n_initial=10,
+    seed=42
+)
+
+# Run optimization
+result = optimizer.optimize()
+
+print(f"Best point: {result.x}")
+print(f"Best value: {result.fun}")
+```
+
+## Using Kriging Surrogate
+
+SpotOptim includes a simplified Kriging (Gaussian Process) surrogate as an alternative to scikit-learn's GaussianProcessRegressor:
+
+```python
+from spotoptim import SpotOptim, Kriging
+
+# Create Kriging surrogate
+kriging = Kriging(
+    noise=1e-6,
+    min_theta=-3.0,
+    max_theta=2.0,
+    seed=42
+)
+
+# Use with SpotOptim
+optimizer = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    surrogate=kriging,  # Use Kriging instead of default GP
+    seed=42
+)
+
+result = optimizer.optimize()
+```
+
+## API Reference
+
+### SpotOptim
+
+**Parameters:**
+- `fun` (callable): Objective function to minimize
+- `bounds` (list of tuples): Bounds for each dimension as [(low, high), ...]
+- `max_iter` (int, default=20): Maximum number of optimization iterations
+- `n_initial` (int, default=10): Number of initial design points
+- `surrogate` (object, optional): Surrogate model (default: GaussianProcessRegressor)
+- `acquisition` (str, default='ei'): Acquisition function ('ei', 'y', 'pi')
+- `var_type` (list of str, optional): Variable types for each dimension
+- `tolerance_x` (float, optional): Minimum distance between points
+- `seed` (int, optional): Random seed for reproducibility
+- `verbose` (bool, default=False): Print progress information
+
+**Methods:**
+- `optimize(X0=None)`: Run optimization, optionally with initial design points
+
+### Kriging
+
+**Parameters:**
+- `noise` (float, optional): Regularization parameter
+- `kernel` (str, default='gauss'): Kernel type
+- `n_theta` (int, optional): Number of theta parameters
+- `min_theta` (float, default=-3.0): Minimum log10(theta) bound
+- `max_theta` (float, default=2.0): Maximum log10(theta) bound
+- `seed` (int, optional): Random seed
+
+**Methods:**
+- `fit(X, y)`: Fit the model to training data
+- `predict(X, return_std=False)`: Predict at new points
+
+## Examples
+
+See the `notebooks/demos.ipynb` for comprehensive examples including:
+1. 2D Rosenbrock function optimization
+2. 6D Rosenbrock with budget constraints
+3. Using Kriging surrogate vs default GP
+
+## Development
+
+```bash
+# Clone repository
+git clone https://github.com/sequential-parameter-optimization/spotoptim.git
+cd spotoptim
+
+# Install with uv
+uv pip install -e .
+
+# Run tests
+uv run pytest tests/
+
+# Build package
+uv build
+```
+
+## License
+
+See LICENSE file.
+
+## References
+
+Based on the SPOT (Sequential Parameter Optimization Toolbox) methodology.

{spotoptim-0.0.3 → spotoptim-0.0.5}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "spotoptim"
-version = "0.0.3"
+version = "0.0.5"
 description = "Add your description here"
 readme = "README.md"
 authors = [
@@ -8,12 +8,9 @@ authors = [
 ]
 requires-python = ">=3.10"
 dependencies = [
-    "fastapi>=0.121.1",
     "numpy>=1.24.3",
     "scipy>=1.10.1",
-    "uvicorn>=0.22.0",
-    "scikit-learn",
-    "spotpython"
+    "scikit-learn>=1.3.0",
 ]
 
 [dependency-groups]

{spotoptim-0.0.3 → spotoptim-0.0.5}/src/spotoptim/SpotOptim.py

@@ -1,10 +1,10 @@
 import numpy as np
 from typing import Callable, Optional, Tuple, List
 from scipy.optimize import OptimizeResult, differential_evolution
+from scipy.stats.qmc import LatinHypercube
 from sklearn.base import BaseEstimator
 from sklearn.gaussian_process import GaussianProcessRegressor
 from sklearn.gaussian_process.kernels import Matern, ConstantKernel
-from spotpython.design.spacefilling import SpaceFilling
 import warnings
 
 
@@ -72,7 +72,7 @@ class SpotOptim(BaseEstimator):
         surrogate: Optional[object] = None,
         acquisition: str = "ei",
         var_type: Optional[list] = None,
-        tolerance_x: float = 1e-6,
+        tolerance_x: Optional[float] = None,
         seed: Optional[int] = None,
         verbose: bool = False,
         warnings_filter: str = "ignore",
@@ -80,6 +80,14 @@ class SpotOptim(BaseEstimator):
 
         warnings.filterwarnings(warnings_filter)
 
+        # small value, converted to float
+        self.eps = np.sqrt(np.spacing(1))
+
+        if tolerance_x is None:
+            self.tolerance_x = self.eps
+        else:
+            self.tolerance_x = tolerance_x
+
         self.fun = fun
         self.bounds = bounds
         self.max_iter = max_iter
@@ -87,7 +95,6 @@ class SpotOptim(BaseEstimator):
         self.surrogate = surrogate
         self.acquisition = acquisition
         self.var_type = var_type
-        self.tolerance_x = tolerance_x
         self.seed = seed
         self.verbose = verbose
 
@@ -113,7 +120,7 @@ class SpotOptim(BaseEstimator):
             )
 
         # Design generator
-        self.design = SpaceFilling(k=self.n_dim, seed=self.seed)
+        self.lhs_sampler = LatinHypercube(d=self.n_dim, seed=self.seed)
 
         # Storage for results
         self.X_ = None
@@ -139,17 +146,22 @@ class SpotOptim(BaseEstimator):
         return y
 
     def _generate_initial_design(self) -> np.ndarray:
-        """Generate initial space-filling design."""
-        X0 = self.design.scipy_lhd(
-            n=self.n_initial, repeats=1, lower=self.lower, upper=self.upper
-        )
+        """Generate initial space-filling design using Latin Hypercube Sampling."""
+        # Generate samples in [0, 1]^d
+        X0_unit = self.lhs_sampler.random(n=self.n_initial)
+
+        # Scale to [lower, upper]
+        X0 = self.lower + X0_unit * (self.upper - self.lower)
+
         return self._repair_non_numeric(X0, self.var_type)
 
     def _fit_surrogate(self, X: np.ndarray, y: np.ndarray) -> None:
         """Fit surrogate model to data."""
         self.surrogate.fit(X, y)
 
-    def _select_new(self, A: np.ndarray, X: np.ndarray, tolerance: float = 0) -> Tuple[np.ndarray, np.ndarray]:
+    def _select_new(
+        self, A: np.ndarray, X: np.ndarray, tolerance: float = 0
+    ) -> Tuple[np.ndarray, np.ndarray]:
         """
         Select rows from A that are not in X.
 
@@ -278,9 +290,9 @@ class SpotOptim(BaseEstimator):
             # If too close, generate random point
             if self.verbose:
                 print("Proposed point too close, generating random point")
-            x_next = self.design.scipy_lhd(
-                n=1, repeats=1, lower=self.lower, upper=self.upper
-            )[0]
+            # Generate a random point using LHS
+            x_next_unit = self.lhs_sampler.random(n=1)[0]
+            x_next = self.lower + x_next_unit * (self.upper - self.lower)
 
         return self._repair_non_numeric(x_next.reshape(1, -1), self.var_type)[0]
 

spotoptim-0.0.5/src/spotoptim/__init__.py

@@ -0,0 +1,11 @@
+"""SpotOptim - Sequential Parameter Optimization."""
+
+from .SpotOptim import SpotOptim
+from .surrogate import Kriging
+
+__version__ = "0.0.3"
+__all__ = ["SpotOptim", "Kriging"]
+
+
+def hello() -> str:
+    return "Hello from spotoptim!"

spotoptim-0.0.5/src/spotoptim/surrogate/README.md

@@ -0,0 +1,149 @@
+# SpotOptim Surrogate Models
+
+This module provides surrogate models for use with the SpotOptim optimizer.
+
+## Kriging Surrogate
+
+The `Kriging` class provides a simplified Gaussian Process (Kriging) surrogate model that can be used as an alternative to scikit-learn's `GaussianProcessRegressor`.
+
+### Features
+
+- **Scikit-learn compatible interface**: Implements `fit()` and `predict()` methods
+- **Automatic hyperparameter optimization**: Uses maximum likelihood estimation
+- **Gaussian (RBF) kernel**: Exponential correlation function
+- **Prediction uncertainty**: Supports `return_std=True` for standard deviations
+- **Reproducible**: Supports random seed for consistent results
+
+### Basic Usage
+
+```python
+import numpy as np
+from spotoptim import SpotOptim, Kriging
+
+# Define objective function
+def sphere(X):
+    X = np.atleast_2d(X)
+    return np.sum(X**2, axis=1)
+
+# Create Kriging surrogate
+kriging = Kriging(
+    noise=1e-6,
+    min_theta=-3.0,
+    max_theta=2.0,
+    seed=42
+)
+
+# Use with SpotOptim
+optimizer = SpotOptim(
+    fun=sphere,
+    bounds=[(-5, 5), (-5, 5)],
+    max_iter=20,
+    n_initial=10,
+    surrogate=kriging,  # Use Kriging surrogate
+    seed=42
+)
+
+result = optimizer.optimize()
+```
+
+### Parameters
+
+- **noise** (float, optional): Regularization parameter (nugget effect). If None, uses sqrt(machine epsilon).
+- **kernel** (str, default='gauss'): Kernel type. Currently only 'gauss' (Gaussian/RBF) is supported.
+- **n_theta** (int, optional): Number of theta parameters. If None, uses k (number of dimensions).
+- **min_theta** (float, default=-3.0): Minimum log10(theta) bound for optimization.
+- **max_theta** (float, default=2.0): Maximum log10(theta) bound for optimization.
+- **seed** (int, optional): Random seed for reproducibility.
+
+### Methods
+
+#### fit(X, y)
+
+Fit the Kriging model to training data.
+
+**Parameters:**
+- `X`: ndarray of shape (n_samples, n_features) - Training input data
+- `y`: ndarray of shape (n_samples,) - Training target values
+
+**Returns:**
+- `self`: Fitted estimator
+
+#### predict(X, return_std=False)
+
+Predict using the Kriging model.
+
+**Parameters:**
+- `X`: ndarray of shape (n_samples, n_features) - Points to predict at
+- `return_std`: bool, default=False - If True, return standard deviations as well
+
+**Returns:**
+- `y_pred`: ndarray of shape (n_samples,) - Predicted values
+- `y_std`: ndarray of shape (n_samples,) - Standard deviations (only if return_std=True)
+
+### Comparison with scikit-learn's GaussianProcessRegressor
+
+| Feature | Kriging | GaussianProcessRegressor |
+|---------|---------|--------------------------|
+| Interface | scikit-learn compatible | Native scikit-learn |
+| Kernel | Gaussian (RBF) | Multiple kernel types |
+| Hyperparameters | Theta (length scales) | Flexible kernel parameters |
+| Dependencies | NumPy, SciPy | NumPy, SciPy, scikit-learn |
+| Optimization | Differential evolution | L-BFGS-B with restarts |
+| Code complexity | Simplified, ~350 lines | Full-featured, complex |
+
+### When to Use Kriging
+
+- You want a **self-contained** surrogate without heavy scikit-learn dependency
+- You need a **simple, interpretable** Gaussian kernel
+- You want **explicit control** over hyperparameter bounds
+- You're working with **moderate-dimensional** problems (< 20 dimensions)
+
+### When to Use Default GP
+
+- You need **multiple kernel types** (Matern, RationalQuadratic, etc.)
+- You want **advanced features** like gradient-based predictions
+- You're working with **very high-dimensional** problems
+- You need **production-tested** robustness
+
+## Example: Comparison
+
+```python
+from spotoptim import SpotOptim, Kriging
+
+def rosenbrock(X):
+    X = np.atleast_2d(X)
+    x, y = X[:, 0], X[:, 1]
+    return (1 - x)**2 + 100 * (y - x**2)**2
+
+bounds = [(-2, 2), (-2, 2)]
+
+# With Kriging
+optimizer_kriging = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    surrogate=Kriging(seed=42),
+    seed=42
+)
+result_kriging = optimizer_kriging.optimize()
+
+# With default GP (no surrogate argument)
+optimizer_gp = SpotOptim(
+    fun=rosenbrock,
+    bounds=bounds,
+    seed=42
+)
+result_gp = optimizer_gp.optimize()
+
+print(f"Kriging result: {result_kriging.fun:.6f}")
+print(f"GP result: {result_gp.fun:.6f}")
+```
+
+## Future Extensions
+
+Planned features for future releases:
+
+- Additional kernel types (Matern, Exponential, etc.)
+- Anisotropic hyperparameters
+- Gradient information
+- Batch predictions
+- Parallel hyperparameter optimization

spotoptim-0.0.5/src/spotoptim/surrogate/__init__.py

@@ -0,0 +1,5 @@
+"""Surrogate models for SpotOptim."""
+
+from .kriging import Kriging
+
+__all__ = ["Kriging"]

spotoptim-0.0.5/src/spotoptim/surrogate/kriging.py

@@ -0,0 +1,360 @@
+"""
+Simplified Kriging surrogate model for SpotOptim.
+
+This is a streamlined version adapted from spotpython.surrogate.kriging
+for use with the SpotOptim optimizer.
+"""
+
+import numpy as np
+from typing import Optional, Tuple, List
+from numpy.linalg import LinAlgError, cholesky, solve
+from scipy.optimize import differential_evolution
+from sklearn.base import BaseEstimator, RegressorMixin
+
+
+class Kriging(BaseEstimator, RegressorMixin):
+    """
+    A simplified Kriging (Gaussian Process) surrogate model for SpotOptim.
+
+    This class provides a scikit-learn compatible interface with fit() and predict()
+    methods, making it suitable for use as a surrogate in SpotOptim.
+
+    Parameters
+    ----------
+    noise : float, optional
+        Regularization parameter (nugget effect). If None, uses sqrt(eps).
+    kernel : str, default='gauss'
+        Kernel type. Currently only 'gauss' (Gaussian/RBF) is supported.
+    n_theta : int, optional
+        Number of theta parameters. If None, uses k (number of dimensions).
+    min_theta : float, default=-3.0
+        Minimum log10(theta) bound for optimization.
+    max_theta : float, default=2.0
+        Maximum log10(theta) bound for optimization.
+    seed : int, optional
+        Random seed for reproducibility.
+
+    Attributes
+    ----------
+    X_ : ndarray of shape (n_samples, n_features)
+        Training data.
+    y_ : ndarray of shape (n_samples,)
+        Training targets.
+    theta_ : ndarray
+        Optimized theta parameters (log10 scale).
+    mu_ : float
+        Mean of the Kriging predictor.
+    sigma2_ : float
+        Variance of the Kriging predictor.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> from spotoptim.surrogate import Kriging
+    >>> X = np.array([[0.0], [0.5], [1.0]])
+    >>> y = np.array([0.0, 0.25, 1.0])
+    >>> model = Kriging()
+    >>> model.fit(X, y)
+    >>> predictions = model.predict(np.array([[0.25], [0.75]]))
+    """
+
+    def __init__(
+        self,
+        noise: Optional[float] = None,
+        kernel: str = "gauss",
+        n_theta: Optional[int] = None,
+        min_theta: float = -3.0,
+        max_theta: float = 2.0,
+        seed: Optional[int] = None,
+    ):
+        self.noise = noise
+        self.kernel = kernel
+        self.n_theta = n_theta
+        self.min_theta = min_theta
+        self.max_theta = max_theta
+        self.seed = seed
+
+        # Fitted attributes
+        self.X_ = None
+        self.y_ = None
+        self.theta_ = None
+        self.mu_ = None
+        self.sigma2_ = None
+        self.U_ = None  # Cholesky factor
+        self.Rinv_one_ = None
+        self.Rinv_r_ = None
+
+    def _get_noise(self) -> float:
+        """Get the noise/regularization parameter."""
+        if self.noise is None:
+            return np.sqrt(np.finfo(float).eps)
+        return self.noise
+
+    def _correlation(self, D: np.ndarray) -> np.ndarray:
+        """
+        Compute correlation from distance matrix using Gaussian kernel.
+
+        Parameters
+        ----------
+        D : ndarray
+            Squared distance matrix.
+
+        Returns
+        -------
+        ndarray
+            Correlation matrix.
+        """
+        if self.kernel == "gauss":
+            return np.exp(-D)
+        else:
+            raise ValueError(f"Unsupported kernel: {self.kernel}")
+
+    def _build_correlation_matrix(self, X: np.ndarray, theta: np.ndarray) -> np.ndarray:
+        """
+        Build correlation matrix R for training data.
+
+        Parameters
+        ----------
+        X : ndarray of shape (n, k)
+            Input data.
+        theta : ndarray of shape (k,)
+            Theta parameters (10^theta used as weights).
+
+        Returns
+        -------
+        ndarray of shape (n, n)
+            Correlation matrix with noise on diagonal.
+        """
+        n = X.shape[0]
+        theta10 = 10.0**theta
+
+        # Compute weighted squared distances
+        R = np.zeros((n, n))
+        for i in range(n):
+            for j in range(i + 1, n):
+                diff = X[i] - X[j]
+                dist = np.sum(theta10 * diff**2)
+                R[i, j] = dist
+                R[j, i] = dist
+
+        # Apply correlation function
+        R = self._correlation(R)
+
+        # Add noise to diagonal
+        noise_val = self._get_noise()
+        np.fill_diagonal(R, 1.0 + noise_val)
+
+        return R
+
+    def _build_correlation_vector(
+        self, x: np.ndarray, X: np.ndarray, theta: np.ndarray
+    ) -> np.ndarray:
+        """
+        Build correlation vector between new point x and training data X.
+
+        Parameters
+        ----------
+        x : ndarray of shape (k,)
+            New point.
+        X : ndarray of shape (n, k)
+            Training data.
+        theta : ndarray of shape (k,)
+            Theta parameters.
+
+        Returns
+        -------
+        ndarray of shape (n,)
+            Correlation vector.
+        """
+        theta10 = 10.0**theta
+        diff = X - x.reshape(1, -1)
+        D = np.sum(theta10 * diff**2, axis=1)
+        return self._correlation(D)
+
+    def _neg_log_likelihood(self, log_theta: np.ndarray) -> float:
+        """
+        Compute negative concentrated log-likelihood.
+
+        Parameters
+        ----------
+        log_theta : ndarray
+            Log10(theta) parameters.
+
+        Returns
+        -------
+        float
+            Negative log-likelihood (to be minimized).
+        """
+        try:
+            n = self.X_.shape[0]
+            y = self.y_.flatten()
+            one = np.ones(n)
+
+            # Build correlation matrix
+            R = self._build_correlation_matrix(self.X_, log_theta)
+
+            # Cholesky decomposition
+            try:
+                U = cholesky(R)
+            except LinAlgError:
+                return 1e10  # Penalty for ill-conditioned matrix
+
+            # Solve for mean and variance
+            Uy = solve(U, y)
+            Uone = solve(U, one)
+
+            Rinv_y = solve(U.T, Uy)
+            Rinv_one = solve(U.T, Uone)
+
+            mu = (one @ Rinv_y) / (one @ Rinv_one)
+            r = y - one * mu
+
+            Ur = solve(U, r)
+            Rinv_r = solve(U.T, Ur)
+
+            sigma2 = (r @ Rinv_r) / n
+
+            if sigma2 <= 0:
+                return 1e10
+
+            # Concentrated log-likelihood
+            log_det_R = 2.0 * np.sum(np.log(np.abs(np.diag(U))))
+            neg_log_like = (n / 2.0) * np.log(sigma2) + 0.5 * log_det_R
+
+            return neg_log_like
+
+        except (LinAlgError, ValueError):
+            return 1e10
+
+    def fit(self, X: np.ndarray, y: np.ndarray) -> "Kriging":
+        """
+        Fit the Kriging model to training data.
+
+        Parameters
+        ----------
+        X : ndarray of shape (n_samples, n_features)
+            Training input data.
+        y : ndarray of shape (n_samples,)
+            Training target values.
+
+        Returns
+        -------
+        self
+            Fitted estimator.
+        """
+        X = np.atleast_2d(X)
+        y = np.asarray(y).flatten()
+
+        if X.ndim != 2:
+            raise ValueError(f"X must be 2-dimensional, got shape {X.shape}")
+        if y.ndim != 1:
+            raise ValueError(f"y must be 1-dimensional, got shape {y.shape}")
+        if X.shape[0] != y.shape[0]:
+            raise ValueError(f"X and y must have same number of samples")
+
+        self.X_ = X
+        self.y_ = y
+        n, k = X.shape
+
+        # Set number of theta parameters
+        if self.n_theta is None:
+            self.n_theta = k
+
+        # Optimize theta via maximum likelihood
+        bounds = [(self.min_theta, self.max_theta)] * self.n_theta
+
+        result = differential_evolution(
+            func=self._neg_log_likelihood,
+            bounds=bounds,
+            seed=self.seed,
+            maxiter=100,
+            atol=1e-6,
+            tol=0.01,
+        )
+
+        self.theta_ = result.x
+
+        # Compute final model parameters
+        one = np.ones(n)
+        R = self._build_correlation_matrix(X, self.theta_)
+
+        try:
+            self.U_ = cholesky(R)
+        except LinAlgError:
+            # Add more regularization if needed
+            R = self._build_correlation_matrix(X, self.theta_)
+            R += np.eye(n) * 1e-8
+            self.U_ = cholesky(R)
+
+        Uy = solve(self.U_, y)
+        Uone = solve(self.U_, one)
+
+        Rinv_y = solve(self.U_.T, Uy)
+        Rinv_one = solve(self.U_.T, Uone)
+
+        self.mu_ = float((one @ Rinv_y) / (one @ Rinv_one))
+
+        r = y - one * self.mu_
+        Ur = solve(self.U_, r)
+        Rinv_r = solve(self.U_.T, Ur)
+
+        self.sigma2_ = float((r @ Rinv_r) / n)
+
+        # Store for prediction
+        self.Rinv_one_ = Rinv_one
+        self.Rinv_r_ = Rinv_r
+
+        return self
+
+    def predict(self, X: np.ndarray, return_std: bool = False) -> np.ndarray:
+        """
+        Predict using the Kriging model.
+
+        Parameters
+        ----------
+        X : ndarray of shape (n_samples, n_features)
+            Points to predict at.
+        return_std : bool, default=False
+            If True, return standard deviations as well.
+
+        Returns
+        -------
+        y_pred : ndarray of shape (n_samples,)
+            Predicted values.
+        y_std : ndarray of shape (n_samples,), optional
+            Standard deviations (only if return_std=True).
+        """
+        X = np.atleast_2d(X)
+
+        if X.ndim == 1:
+            X = X.reshape(1, -1)
+
+        if X.shape[1] != self.X_.shape[1]:
+            raise ValueError(
+                f"X has {X.shape[1]} features, expected {self.X_.shape[1]}"
+            )
+
+        n_pred = X.shape[0]
+        predictions = np.zeros(n_pred)
+
+        if return_std:
+            std_devs = np.zeros(n_pred)
+
+        for i, x in enumerate(X):
+            # Build correlation vector
+            psi = self._build_correlation_vector(x, self.X_, self.theta_)
+
+            # Predict mean
+            predictions[i] = self.mu_ + psi @ self.Rinv_r_
+
+            if return_std:
+                # Predict variance
+                Upsi = solve(self.U_, psi)
+                psi_Rinv_psi = psi @ solve(self.U_.T, Upsi)
+
+                variance = self.sigma2_ * (1.0 + self._get_noise() - psi_Rinv_psi)
+                std_devs[i] = np.sqrt(max(0.0, variance))
+
+        if return_std:
+            return predictions, std_devs
+        return predictions

spotoptim-0.0.3/PKG-INFO

@@ -1,15 +0,0 @@
-Metadata-Version: 2.3
-Name: spotoptim
-Version: 0.0.3
-Summary: Add your description here
-Author: bartzbeielstein
-Author-email: bartzbeielstein <32470350+bartzbeielstein@users.noreply.github.com>
-Requires-Dist: fastapi>=0.121.1
-Requires-Dist: numpy>=1.24.3
-Requires-Dist: scipy>=1.10.1
-Requires-Dist: uvicorn>=0.22.0
-Requires-Dist: scikit-learn
-Requires-Dist: spotpython
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-

spotoptim-0.0.3/src/spotoptim/__init__.py

@@ -1,2 +0,0 @@
-def hello() -> str:
-    return "Hello from spotoptim!"