better-optimize 0.1.2__tar.gz → 0.1.4__tar.gz

better_optimize-0.1.4/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.1
+Name: better_optimize
+Version: 0.1.4
+Summary: A drop-in replacement for scipy optimize functions with quality of life improvements
+License: MIT License
+Project-URL: Homepage, https://github.com/jessegrabowski/better_optimize
+Project-URL: Repository, https://github.com/jessegrabowski/better_optimize.git
+Project-URL: Issues, https://github.com/jessegrabowski/better_optimize/issues
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Requires-Python: <3.14,>=3.10
+Description-Content-Type: text/markdown
+Provides-Extra: tests
+License-File: LICENSE
+
+# Better Optimization!
+
+`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
+include:
+
+- Progress bar!
+- Early stopping!
+- Better propagation of common arguments (`maxiters`, `tol`)!
+
+## Installation
+
+To install `better_optimize`, simply use conda:
+
+```bash
+conda install -c conda-forge better_optimize
+```
+
+Or, if you prefer pip:
+
+```bash
+pip install better_optimize
+```
+
+## What does `better_optimize` provide over basic scipy?
+
+### 1. Progress Bars
+
+All optimization routines in `better_optimize` can display a rich, informative progress bar using the [rich](https://github.com/Textualize/rich) library. This includes:
+
+- Iteration counts, elapsed time, and objective values.
+- Gradient and Hessian norms (when available).
+- Separate progress bars for global (basinhopping) and local (minimizer) steps.
+- Toggleable display for headless or script environments.
+
+### 2. Flat and Generalized Keyword Arguments
+
+- No more nested `options` dictionaries! You can pass `tol`, `maxiter`, and other common options directly as top-level keyword arguments.
+- `better_optimize` automatically sorts and promotes these arguments to the correct place for each optimizer.
+- Generalizes argument handling: always provides `tol` and `maxiter` (or their equivalents) to the optimizer, even if you forget.
+
+### 3. Argument Checking and Validation
+
+- Automatic checking of provided gradient (`jac`), Hessian (`hess`), and Hessian-vector (`hessp`) functions.
+- Warns if you provide unnecessary or unused arguments for a given method.
+- Detects and handles fused objective functions (e.g., functions returning `(loss, grad)` or `(loss, grad, hess)` tuples).
+- Ensures that the correct function signatures and return types are used for each optimizer.
+
+### 4. LRUCache1 for Fused Functions
+
+- Provides an `LRUCache1` utility to cache the results of expensive objective/gradient/Hessian computations.
+- Especially useful for triple-fused functions that return value, gradient, and Hessian together, avoiding redundant computation.
+- Totally invisible -- just pass a function with 3 return values. Seamlessly integrated into the optimization workflow.
+
+### 5. Robust Basin-Hopping with Failure Tolerance
+
+- Enhanced `basinhopping` implementation allows you to continue even if the local minimizer fails.
+- Optionally accepts and stores failed minimizer results if they improve the global minimum.
+- Useful for noisy or non-smooth objective functions where local minimization may occasionally fail.
+
+---
+
+## Example Usage
+
+### Simple Example
+
+```python
+from better_optimize import minimize
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+result = minimize(
+    rosenbrock,
+    x0=[-1, 2],
+    method="L-BFGS-B",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+```shell
+  Minimizing                                         Elapsed   Iteration   Objective    ||grad||
+ ──────────────────────────────────────────────────────────────────────────────────────────────────
+  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0:00:00   721/721     0.34271757   0.92457651
+```
+
+The result object is a standard `OptimizeResult` from `scipy.optimize`, so there are no surprises there!
+
+### Triple-Fused Function using Pytensor
+
+```python
+from better_optimize import minimize
+import pytensor.tensor as pt
+from pytensor import function
+import numpy as np
+
+x = pt.vector('x')
+value = pt.sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+grad = pt.grad(value, x)
+hess = pt.hessian(value, x)
+
+fused_fn = function([x], [value, grad, hess])
+x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
+
+result = minimize(
+    fused_fn, # No need to set flags separately, `better_optimize` handles it!
+    x0=x0,
+    method="Newton-CG",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+Many sub-computations are repeated between the objective, gradient, and hessian functions. Scipy allows you to pass a
+fused value_and_grad function, but `better_optimize` also lets you pass a triple-fused value_grad_and_hess function.
+This avoids redundant computation and speeds up the optimization process.
+
+
+## Contributing
+
+We welcome contributions! If you find a bug, have a feature request, or want to improve the documentation, please open
+an issue or submit a pull request on GitHub.

better_optimize-0.1.4/README.md

@@ -0,0 +1,124 @@
+# Better Optimization!
+
+`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
+include:
+
+- Progress bar!
+- Early stopping!
+- Better propagation of common arguments (`maxiters`, `tol`)!
+
+## Installation
+
+To install `better_optimize`, simply use conda:
+
+```bash
+conda install -c conda-forge better_optimize
+```
+
+Or, if you prefer pip:
+
+```bash
+pip install better_optimize
+```
+
+## What does `better_optimize` provide over basic scipy?
+
+### 1. Progress Bars
+
+All optimization routines in `better_optimize` can display a rich, informative progress bar using the [rich](https://github.com/Textualize/rich) library. This includes:
+
+- Iteration counts, elapsed time, and objective values.
+- Gradient and Hessian norms (when available).
+- Separate progress bars for global (basinhopping) and local (minimizer) steps.
+- Toggleable display for headless or script environments.
+
+### 2. Flat and Generalized Keyword Arguments
+
+- No more nested `options` dictionaries! You can pass `tol`, `maxiter`, and other common options directly as top-level keyword arguments.
+- `better_optimize` automatically sorts and promotes these arguments to the correct place for each optimizer.
+- Generalizes argument handling: always provides `tol` and `maxiter` (or their equivalents) to the optimizer, even if you forget.
+
+### 3. Argument Checking and Validation
+
+- Automatic checking of provided gradient (`jac`), Hessian (`hess`), and Hessian-vector (`hessp`) functions.
+- Warns if you provide unnecessary or unused arguments for a given method.
+- Detects and handles fused objective functions (e.g., functions returning `(loss, grad)` or `(loss, grad, hess)` tuples).
+- Ensures that the correct function signatures and return types are used for each optimizer.
+
+### 4. LRUCache1 for Fused Functions
+
+- Provides an `LRUCache1` utility to cache the results of expensive objective/gradient/Hessian computations.
+- Especially useful for triple-fused functions that return value, gradient, and Hessian together, avoiding redundant computation.
+- Totally invisible -- just pass a function with 3 return values. Seamlessly integrated into the optimization workflow.
+
+### 5. Robust Basin-Hopping with Failure Tolerance
+
+- Enhanced `basinhopping` implementation allows you to continue even if the local minimizer fails.
+- Optionally accepts and stores failed minimizer results if they improve the global minimum.
+- Useful for noisy or non-smooth objective functions where local minimization may occasionally fail.
+
+---
+
+## Example Usage
+
+### Simple Example
+
+```python
+from better_optimize import minimize
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+result = minimize(
+    rosenbrock,
+    x0=[-1, 2],
+    method="L-BFGS-B",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+```shell
+  Minimizing                                         Elapsed   Iteration   Objective    ||grad||
+ ──────────────────────────────────────────────────────────────────────────────────────────────────
+  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0:00:00   721/721     0.34271757   0.92457651
+```
+
+The result object is a standard `OptimizeResult` from `scipy.optimize`, so there are no surprises there!
+
+### Triple-Fused Function using Pytensor
+
+```python
+from better_optimize import minimize
+import pytensor.tensor as pt
+from pytensor import function
+import numpy as np
+
+x = pt.vector('x')
+value = pt.sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+grad = pt.grad(value, x)
+hess = pt.hessian(value, x)
+
+fused_fn = function([x], [value, grad, hess])
+x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
+
+result = minimize(
+    fused_fn, # No need to set flags separately, `better_optimize` handles it!
+    x0=x0,
+    method="Newton-CG",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+Many sub-computations are repeated between the objective, gradient, and hessian functions. Scipy allows you to pass a
+fused value_and_grad function, but `better_optimize` also lets you pass a triple-fused value_grad_and_hess function.
+This avoids redundant computation and speeds up the optimization process.
+
+
+## Contributing
+
+We welcome contributions! If you find a bug, have a feature request, or want to improve the documentation, please open
+an issue or submit a pull request on GitHub.

{better_optimize-0.1.2 → better_optimize-0.1.4}/better_optimize/_version.py

@@ -8,11 +8,11 @@ import json
 
 version_json = '''
 {
- "date": "2025-05-14T08:34:39+0200",
+ "date": "2025-06-28T08:22:47+0800",
  "dirty": false,
  "error": null,
- "full-revisionid": "b9f19e0aad67eca088085f337a4627b537f71e3c",
- "version": "0.1.2"
+ "full-revisionid": "323b00a655190dbb83a66ff9c7ce55df30db16c4",
+ "version": "0.1.4"
 }
 '''  # END VERSION_JSON
 

{better_optimize-0.1.2 → better_optimize-0.1.4}/better_optimize/basinhopping.py

@@ -22,7 +22,13 @@ from scipy.optimize._basinhopping import (
 )
 
 from better_optimize.minimize import minimize
-from better_optimize.utilities import ToggleableProgress, check_f_is_fused
+from better_optimize.utilities import (
+    LRUCache1,
+    ToggleableProgress,
+    check_f_is_fused_minimize,
+    determine_maxiter,
+    validate_provided_functions_minimize,
+)
 
 
 def initialize_progress_bar(progressbar, use_jac=True, use_hess=False):
@@ -166,12 +172,36 @@ def basinhopping(
     # set up minimizer
     if minimizer_kwargs is None:
         minimizer_kwargs = dict()
+    minimizer_kwargs = minimizer_kwargs.copy()  # avoid mutating caller's dict
+
+    has_fused_f_and_grad, has_fused_f_grad_hess = check_f_is_fused_minimize(
+        func, x0, minimizer_kwargs.get("args", ())
+    )
+
+    jac = minimizer_kwargs.pop("jac", None)
+    hess = minimizer_kwargs.pop("hess", None)
+    hessp = minimizer_kwargs.pop("hessp", None)
+    method = minimizer_kwargs.pop("method", "L-BFGS-B")
+    args = minimizer_kwargs.pop("args", ())
+
+    # Use the validation helper to determine which functions to use
+    use_jac, use_hess, use_hessp = validate_provided_functions_minimize(
+        method,
+        jac,
+        hess,
+        hessp,
+        has_fused_f_and_grad,
+        has_fused_f_grad_hess,
+        verbose=verbose,
+    )
 
-    has_fused_f_and_grad = check_f_is_fused(func, x0, minimizer_kwargs.get("args", ()))
-    if has_fused_f_and_grad:
-        minimizer_kwargs.pop("jac", None)
+    f_returns_list = has_fused_f_and_grad or has_fused_f_grad_hess
+    f_cached = LRUCache1(func, f_returns_list=f_returns_list, copy_x=False, dtype=x0.dtype)
+
+    # If triple-fused, use hess from cache
+    if has_fused_f_grad_hess:
+        hess = f_cached.hess
 
-    use_jac = has_fused_f_and_grad or minimizer_kwargs.get("jac", False)
     progress = initialize_progress_bar(progressbar, use_jac=use_jac, use_hess=False)
 
     if progressbar:
@@ -188,6 +218,11 @@ def basinhopping(
         hess_norm=0.0,
     )
 
+    n_vars = x0.size if hasattr(x0, "size") else len(x0)
+    if "options" not in minimizer_kwargs:
+        minimizer_kwargs["options"] = {}
+    minimizer_maxiter, minimizer_kwargs = determine_maxiter(minimizer_kwargs, method, n_vars)
+
     minimize_task = progress.add_task(
         description="Minimize",
         name="Minimize",
@@ -196,18 +231,28 @@ def basinhopping(
         f_value=0.0,
         grad_norm=0.0,
         hess_norm=0.0,
+        total=minimizer_maxiter,
     )
 
-    wrapped_minimizer = MinimizerWrapper(
-        minimize,
-        func,
-        progressbar=progress,
-        progress_task=minimize_task,
-        verbose=verbose,
-        **minimizer_kwargs,
-    )
+    def wrapped_minimize(x0_inner, *args, **kwargs):
+        # Always keep the minimizer progress bar's total fixed at the requested maxiter
+        progress.update(minimize_task, total=minimizer_maxiter)
+        return minimize(
+            method=method,
+            f=f_cached.value_and_grad if has_fused_f_and_grad else f_cached.value,
+            x0=x0_inner,
+            args=args,
+            jac=None if not use_jac else (True if has_fused_f_and_grad else jac),
+            hess=None if not use_hess else hess,
+            hessp=None if not use_hessp else hessp,
+            progressbar=progress,
+            progress_task=minimize_task,
+            verbose=verbose,
+            **minimizer_kwargs,
+        )
+
+    wrapped_minimizer = MinimizerWrapper(wrapped_minimize)
 
-    # set up step-taking algorithm
     if take_step is not None:
         if not callable(take_step):
             raise TypeError("take_step must be callable")
@@ -268,9 +313,19 @@ def basinhopping(
 
     grad_norm_at_min = 0.0
     if use_jac:
-        _, grad_val = func(bh.x)
+        grad_val = f_cached.grad(bh.x, *args)
         grad_norm_at_min = np.linalg.norm(grad_val)
 
+    # Update the progress bar with initial values
+    progress.update(
+        bh_task,
+        advance=1,
+        step_size=take_step_wrapped.takestep.stepsize,
+        target_accept=take_step_wrapped.target_accept_rate,
+        f_value=bh.storage.get_lowest().fun,
+        grad_norm=grad_norm_at_min,
+    )
+
     with progress:
         for i in range(niter):
             if progressbar:

{better_optimize-0.1.2 → better_optimize-0.1.4}/better_optimize/minimize.py

@@ -10,7 +10,8 @@ from scipy.sparse.linalg import LinearOperator
 
 from better_optimize.constants import minimize_method
 from better_optimize.utilities import (
-    check_f_is_fused,
+    LRUCache1,
+    check_f_is_fused_minimize,
     determine_maxiter,
     determine_tolerance,
     kwargs_to_options,
@@ -69,11 +70,18 @@ def minimize(
         Optimization result
 
     """
-    has_fused_f_and_grad = check_f_is_fused(f, x0, args)
-    validate_provided_functions_minimize(
-        method, jac, hess, hessp, has_fused_f_and_grad, verbose=verbose
+    has_fused_f_and_grad, has_fused_f_grad_hess = check_f_is_fused_minimize(f, x0, args)
+
+    use_grad, use_hess, use_hessp = validate_provided_functions_minimize(
+        method, jac, hess, hessp, has_fused_f_and_grad, has_fused_f_grad_hess, verbose=verbose
     )
 
+    f_returns_list = has_fused_f_and_grad or has_fused_f_grad_hess
+    f_cached = LRUCache1(f, f_returns_list=f_returns_list, copy_x=False, dtype=x0.dtype)
+
+    if has_fused_f_grad_hess:
+        hess = f_cached.hess
+
     options = optimizer_kwargs.pop("options", {})
     optimizer_kwargs["options"] = options
 
@@ -83,12 +91,11 @@ def minimize(
 
     # Test hessian function -- if it returns a LinearOperator, it can't be used inside the wrapper
     args = () if args is None else args
-    use_hess = hess is not None and not isinstance(hess(x0, *args), LinearOperator)
-    use_hessp = hessp is not None
+    use_hess = use_hess and not isinstance(hess(x0, *args), LinearOperator)
 
     objective = ObjectiveWrapper(
         maxeval=maxiter,
-        f=f,
+        f=f_cached.value_and_grad if has_fused_f_and_grad else f_cached.value,
         jac=jac,
         hess=hess if use_hess else None,
         args=args,

{better_optimize-0.1.2 → better_optimize-0.1.4}/better_optimize/root.py

@@ -8,7 +8,8 @@ from scipy.optimize import root as sp_root
 
 from better_optimize.constants import root_method
 from better_optimize.utilities import (
-    check_f_is_fused,
+    LRUCache1,
+    check_f_is_fused_root,
     determine_maxiter,
     determine_tolerance,
     kwargs_to_jac_options,
@@ -65,10 +66,10 @@ def root(
         Optimization result
 
     """
-    has_fused_f_and_grad = check_f_is_fused(f, x0, args)
-    validate_provided_functions_root(
-        method, jac, has_fused_f_and_grad, has_fused_f_and_grad, verbose=verbose
-    )
+    has_fused_f_and_grad = check_f_is_fused_root(f, x0, args)
+    validate_provided_functions_root(method, f, jac, has_fused_f_and_grad, verbose=verbose)
+
+    f_cached = LRUCache1(f, f_returns_list=has_fused_f_and_grad, copy_x=True, dtype=x0.dtype)
 
     options = optimizer_kwargs.pop("options", {})
     optimizer_kwargs["options"] = options
@@ -80,7 +81,7 @@ def root(
 
     objective = ObjectiveWrapper(
         maxeval=maxiter,
-        f=f,
+        f=f_cached.value_and_grad if has_fused_f_and_grad else f_cached.value,
         jac=jac,
         args=args,
         progressbar=progressbar,

{better_optimize-0.1.2 → better_optimize-0.1.4}/better_optimize/utilities.py

@@ -154,19 +154,36 @@ def validate_provided_functions_minimize(
     f_hess: Callable[[np.ndarray], np.ndarray] | None,
     f_hessp: Callable[[np.ndarray], np.ndarray] | None,
     has_fused_f_and_grad: bool,
+    has_fused_f_grad_hess: bool,
     verbose=True,
-) -> None:
+) -> tuple[bool, bool, bool]:
     has_grad, has_hess, has_hessp = map(lambda f: f is not None, [f_grad, f_hess, f_hessp])
     uses_grad, uses_hess, uses_hessp, *_ = MINIMIZE_MODE_KWARGS[method].values()
 
-    if has_fused_f_and_grad and has_grad:
-        _log.warning(
-            "Objective function returns a tuple (interpreted as (loss, gradient), but a gradient function was "
-            "also provided. The gradient function will be ignored."
-        )
+    # Handle fused outputs first
+    if has_fused_f_grad_hess:
+        if verbose and (has_grad or has_hess):
+            _log.warning(
+                "Objective function returns a tuple (interpreted as (loss, gradient, hessian)), "
+                "but a gradient or hessian function was also provided. The gradient and hessian functions will be ignored."
+            )
+        if verbose and has_hessp:
+            _log.warning(
+                "Objective function returns a tuple (interpreted as (loss, gradient, hessian)), "
+                "but a hessian-vector product function was also provided. The hessian-vector product function will be ignored."
+            )
+        # Triple-fused disables external grad/hess/hessp
+        has_grad = True
+        has_hess = True
+        has_hessp = False
 
     elif has_fused_f_and_grad:
-        has_grad = True
+        if verbose and has_grad:
+            _log.warning(
+                "Objective function returns a tuple (interpreted as (loss, gradient)), "
+                "but a gradient function was also provided. The gradient function will be ignored."
+            )
+        has_grad = True  # fused (loss, grad) disables external grad
 
     if method not in MINIMIZE_MODE_KWARGS:
         raise ValueError(
@@ -179,31 +196,33 @@ def validate_provided_functions_minimize(
             "except trust-exact and dogleg, use_hessp is preferred."
         )
 
-    if has_grad and not uses_grad and verbose:
+    if verbose and (has_grad and not uses_grad):
         _log.warning(
             f"Gradient provided but not used by method {method}. Gradients will still be evaluated at each "
-            f"optimzer step and the norm will be reported as a diagnositc. For large problems, this may be "
+            f"optimizer step and the norm will be reported as a diagnostic. For large problems, this may be "
             f"computationally intensive."
         )
 
-    if (has_hess and not uses_hess) or (has_hessp and not uses_hessp) and verbose:
+    if verbose and ((has_hess and not uses_hess) or (has_hessp and not uses_hessp)):
         _log.warning(
-            f"Hessian provided but not used by method {method}. The full hessian will still be evaluated at "
-            f"each optimzer step and the norm will be reported as a diagnositc. For large problems, this may "
-            f"be computationally intensive."
+            f"Hessian or Hessian-vector product provided but not used by method {method}. "
+            f"The full Hessian or Hessian-vector product will still be evaluated at each optimizer step and "
+            f"the norm will be reported as a diagnostic. For large problems, this may be computationally intensive."
         )
 
-    if has_hess and not has_hessp and uses_hessp and uses_hess and verbose:
+    if verbose and (has_hess and not has_hessp and uses_hessp and uses_hess):
         _log.warning(
-            f"You provided a function to compute the full hessian, but method {method} allows the use of a "
-            f"hessian-vector product instead. Consider passing hessp instead -- this may be significantly "
+            f"You provided a function to compute the full Hessian, but method {method} allows the use of a "
+            f"Hessian-vector product instead. Consider passing hessp instead -- this may be significantly "
             f"more efficient."
         )
 
+    return has_grad, has_hess, has_hessp
+
 
 def validate_provided_functions_root(
     method: root_method, f, jac, has_fused_f_and_grad: bool, verbose: bool = True
-):
+) -> bool:
     has_jac = jac is not None
     info_dict = get_option_kwargs(method)
     uses_jac = info_dict["uses_jac"]
@@ -224,19 +243,87 @@ def validate_provided_functions_root(
             f"computationally intensive."
         )
 
+    return has_jac
+
 
-def check_f_is_fused(f, x0, args):
+def check_f_is_fused_minimize(f, x0, args) -> tuple[bool, bool]:
+    """
+    Check if the minimize objective function returns fused outputs (value, grad[, hess]).
+    Returns (is_fused, has_hess).
+    """
     args = () if args is None else args
     output = f(x0, *args)
+
+    if not isinstance(output, tuple | list):
+        return (False, False)
+
+    if len(output) == 2:
+        value, grad = output
+        hess = None
+        ret_val = (True, False)
+    elif len(output) == 3:
+        value, grad, hess = output
+        ret_val = (True, True)
+    else:
+        raise ValueError(
+            "Objective function should return either a scalar loss, a two-tuple of (loss, gradient), "
+            "or three-tuple of (loss, gradient, hessian)."
+        )
+
+    if not (
+        np.isscalar(value)
+        or (hasattr(value, "shape") and (value.shape == () or value.shape == (1,)))
+    ):
+        raise ValueError(
+            "First value returned by the objective function must be a scalar (float or 0-d array)."
+        )
+
+    if not (hasattr(grad, "ndim") and grad.ndim == 1):
+        raise ValueError(
+            "Second value returned by the objective function must be a 1d array representing the gradient."
+        )
+
+    if hess is not None:
+        if not (hasattr(hess, "ndim") and hess.ndim == 2):
+            raise ValueError(
+                "Third value returned by the objective function must be a 2d array representing the Hessian."
+            )
+
+    return ret_val
+
+
+def check_f_is_fused_root(f, x0, args) -> bool:
+    """
+    Check if the root objective function returns fused outputs (value[, jac]), and returns True if it does.
+    """
+    args = () if args is None else args
+    output = f(x0, *args)
+
     if not isinstance(output, tuple | list):
         return False
 
-    # If the output is a tuple, it should be length 2 (returning the value and the grad).If not, something is wrong
-    if len(output) != 2:
+    if len(output) == 2:
+        value, jac = output
+        ret_val = True
+    elif len(output) == 1:
+        value = output[0]
+        jac = None
+        ret_val = True
+    else:
         raise ValueError(
-            "Objective function should return either a scalar loss or a two-tuple of (loss, gradient)"
+            "Objective function should return either a 1d array or a two-tuple of (value, jacobian)."
         )
-    return True
+
+    if not (hasattr(value, "ndim") and value.ndim == 1):
+        raise ValueError("First value returned by the objective function must be a 1d array.")
+
+    if jac is not None:
+        if not (hasattr(jac, "ndim") and jac.ndim == 2):
+            raise ValueError(
+                "Second value returned by the objective function must be a 2d array representing the Jacobian."
+            )
+
+    return ret_val
 
 
 def determine_maxiter(
@@ -314,3 +401,102 @@ def determine_tolerance(optimizer_kwargs: dict, method: minimize_method | root_m
             optimizer_kwargs["options"][tol_name] = tol
 
     return optimizer_kwargs
+
+
+class LRUCache1:
+    """
+    Simple LRU cache with a memory size of 1.
+
+    This cache is only usable for a function that takes a single input `x` and returns a single output. The
+    function can also take any number of additional arguments `*args`, but these are assumed to be constant
+    between function calls.
+
+    The purpose of this cache is to allow for Hessian computation to be reused when calling scipy.optimize functions.
+    It is very often the case that some sub-computations are repeated between the objective, gradient, and hessian
+    functions, but by default scipy only allows for the objective and gradient to be fused.
+
+    By using this cache, all 3 functions can be fused, which can significantly speed up the optimization process for
+    expensive functions.
+    """
+
+    def __init__(
+        self, fn, f_returns_list: bool = False, copy_x: bool = False, dtype: str | None = None
+    ):
+        self.fn = fn
+        self.last_x = None
+        self.last_result = None
+        self.copy_x = copy_x
+        self.f_returns_list = f_returns_list
+
+        # Scipy does not respect dtypes *at all*, so we have to force it ourselves.
+        self.dtype = dtype
+
+        self.cache_hits = 0
+        self.cache_misses = 0
+
+        self.value_calls = 0
+        self.grad_calls = 0
+        self.value_and_grad_calls = 0
+        self.hess_calls = 0
+
+    def __call__(self, x, *args):
+        """
+        Call the cached function with the given input `x` and additional arguments `*args`.
+
+        If the input `x` is the same as the last input, return the cached result. Otherwise update the cache with the
+        new input and result.
+        """
+        x = x.astype(self.dtype)
+
+        if self.last_result is None or not (x == self.last_x).all():
+            self.cache_misses += 1
+
+            # scipy.optimize.root changes x in place, so the cache has to copy it, otherwise we get false
+            # cache hits and optimization always fails.
+            if self.copy_x:
+                x = x.copy()
+            self.last_x = x
+
+            result = self.fn(x, *args)
+            self.last_result = result
+
+            return result
+
+        else:
+            self.cache_hits += 1
+            return self.last_result
+
+    def value(self, x, *args):
+        self.value_calls += 1
+        if not self.f_returns_list:
+            return self(x, *args)
+        else:
+            return self(x, *args)[0]
+
+    def grad(self, x, *args):
+        self.grad_calls += 1
+        return self(x, *args)[1]
+
+    def value_and_grad(self, x, *args):
+        self.value_and_grad_calls += 1
+        return self(x, *args)[:2]
+
+    def hess(self, x, *args):
+        self.hess_calls += 1
+        return self(x, *args)[-1]
+
+    def report(self):
+        _log.info(f"Value and Grad calls: {self.value_and_grad_calls}")
+        _log.info(f"Hess Calls: {self.hess_calls}")
+        _log.info(f"Hits: {self.cache_hits}")
+        _log.info(f"Misses: {self.cache_misses}")
+
+    def clear_cache(self):
+        self.last_x = None
+        self.last_result = None
+        self.cache_hits = 0
+        self.cache_misses = 0
+        self.value_calls = 0
+        self.grad_calls = 0
+        self.value_and_grad_calls = 0
+        self.hess_calls = 0

better_optimize-0.1.4/better_optimize.egg-info/PKG-INFO

@@ -0,0 +1,146 @@
+Metadata-Version: 2.1
+Name: better-optimize
+Version: 0.1.4
+Summary: A drop-in replacement for scipy optimize functions with quality of life improvements
+License: MIT License
+Project-URL: Homepage, https://github.com/jessegrabowski/better_optimize
+Project-URL: Repository, https://github.com/jessegrabowski/better_optimize.git
+Project-URL: Issues, https://github.com/jessegrabowski/better_optimize/issues
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Requires-Python: <3.14,>=3.10
+Description-Content-Type: text/markdown
+Provides-Extra: tests
+License-File: LICENSE
+
+# Better Optimization!
+
+`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
+include:
+
+- Progress bar!
+- Early stopping!
+- Better propagation of common arguments (`maxiters`, `tol`)!
+
+## Installation
+
+To install `better_optimize`, simply use conda:
+
+```bash
+conda install -c conda-forge better_optimize
+```
+
+Or, if you prefer pip:
+
+```bash
+pip install better_optimize
+```
+
+## What does `better_optimize` provide over basic scipy?
+
+### 1. Progress Bars
+
+All optimization routines in `better_optimize` can display a rich, informative progress bar using the [rich](https://github.com/Textualize/rich) library. This includes:
+
+- Iteration counts, elapsed time, and objective values.
+- Gradient and Hessian norms (when available).
+- Separate progress bars for global (basinhopping) and local (minimizer) steps.
+- Toggleable display for headless or script environments.
+
+### 2. Flat and Generalized Keyword Arguments
+
+- No more nested `options` dictionaries! You can pass `tol`, `maxiter`, and other common options directly as top-level keyword arguments.
+- `better_optimize` automatically sorts and promotes these arguments to the correct place for each optimizer.
+- Generalizes argument handling: always provides `tol` and `maxiter` (or their equivalents) to the optimizer, even if you forget.
+
+### 3. Argument Checking and Validation
+
+- Automatic checking of provided gradient (`jac`), Hessian (`hess`), and Hessian-vector (`hessp`) functions.
+- Warns if you provide unnecessary or unused arguments for a given method.
+- Detects and handles fused objective functions (e.g., functions returning `(loss, grad)` or `(loss, grad, hess)` tuples).
+- Ensures that the correct function signatures and return types are used for each optimizer.
+
+### 4. LRUCache1 for Fused Functions
+
+- Provides an `LRUCache1` utility to cache the results of expensive objective/gradient/Hessian computations.
+- Especially useful for triple-fused functions that return value, gradient, and Hessian together, avoiding redundant computation.
+- Totally invisible -- just pass a function with 3 return values. Seamlessly integrated into the optimization workflow.
+
+### 5. Robust Basin-Hopping with Failure Tolerance
+
+- Enhanced `basinhopping` implementation allows you to continue even if the local minimizer fails.
+- Optionally accepts and stores failed minimizer results if they improve the global minimum.
+- Useful for noisy or non-smooth objective functions where local minimization may occasionally fail.
+
+---
+
+## Example Usage
+
+### Simple Example
+
+```python
+from better_optimize import minimize
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+result = minimize(
+    rosenbrock,
+    x0=[-1, 2],
+    method="L-BFGS-B",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+```shell
+  Minimizing                                         Elapsed   Iteration   Objective    ||grad||
+ ──────────────────────────────────────────────────────────────────────────────────────────────────
+  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0:00:00   721/721     0.34271757   0.92457651
+```
+
+The result object is a standard `OptimizeResult` from `scipy.optimize`, so there are no surprises there!
+
+### Triple-Fused Function using Pytensor
+
+```python
+from better_optimize import minimize
+import pytensor.tensor as pt
+from pytensor import function
+import numpy as np
+
+x = pt.vector('x')
+value = pt.sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+grad = pt.grad(value, x)
+hess = pt.hessian(value, x)
+
+fused_fn = function([x], [value, grad, hess])
+x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
+
+result = minimize(
+    fused_fn, # No need to set flags separately, `better_optimize` handles it!
+    x0=x0,
+    method="Newton-CG",
+    tol=1e-6,
+    maxiter=1000,
+    progressbar=True,  # Show a rich progress bar!
+)
+```
+
+Many sub-computations are repeated between the objective, gradient, and hessian functions. Scipy allows you to pass a
+fused value_and_grad function, but `better_optimize` also lets you pass a triple-fused value_grad_and_hess function.
+This avoids redundant computation and speeds up the optimization process.
+
+
+## Contributing
+
+We welcome contributions! If you find a bug, have a feature request, or want to improve the documentation, please open
+an issue or submit a pull request on GitHub.

better_optimize-0.1.2/PKG-INFO

@@ -1,30 +0,0 @@
-Metadata-Version: 2.1
-Name: better_optimize
-Version: 0.1.2
-Summary: A drop-in replacement for scipy optimize functions with quality of life improvements
-License: MIT License
-Project-URL: Homepage, https://github.com/jessegrabowski/better_optimize
-Project-URL: Repository, https://github.com/jessegrabowski/better_optimize.git
-Project-URL: Issues, https://github.com/jessegrabowski/better_optimize/issues
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Scientific/Engineering
-Classifier: Topic :: Scientific/Engineering :: Mathematics
-Requires-Python: <3.14,>=3.10
-Description-Content-Type: text/markdown
-Provides-Extra: tests
-License-File: LICENSE
-
-# Better Optimization!
-
-`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
-include:
-
-- Progress bar!
-- Early stopping!
-- Better propagation of common arguments (`maxiters`, `tol`)!

better_optimize-0.1.2/README.md

@@ -1,8 +0,0 @@
-# Better Optimization!
-
-`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
-include:
-
-- Progress bar!
-- Early stopping!
-- Better propagation of common arguments (`maxiters`, `tol`)!

better_optimize-0.1.2/better_optimize.egg-info/PKG-INFO

@@ -1,30 +0,0 @@
-Metadata-Version: 2.1
-Name: better-optimize
-Version: 0.1.2
-Summary: A drop-in replacement for scipy optimize functions with quality of life improvements
-License: MIT License
-Project-URL: Homepage, https://github.com/jessegrabowski/better_optimize
-Project-URL: Repository, https://github.com/jessegrabowski/better_optimize.git
-Project-URL: Issues, https://github.com/jessegrabowski/better_optimize/issues
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Science/Research
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Scientific/Engineering
-Classifier: Topic :: Scientific/Engineering :: Mathematics
-Requires-Python: <3.14,>=3.10
-Description-Content-Type: text/markdown
-Provides-Extra: tests
-License-File: LICENSE
-
-# Better Optimization!
-
-`better_optimize` is a friendlier front-end to scipy's `optimize.minimize` and `optimize.root` functions. Features
-include:
-
-- Progress bar!
-- Early stopping!
-- Better propagation of common arguments (`maxiters`, `tol`)!