better-optimize 0.4.2__tar.gz → 0.5.0__tar.gz

{better_optimize-0.4.2 → better_optimize-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: better_optimize
-Version: 0.4.2
+Version: 0.5.0
 Summary: A drop-in replacement for scipy optimize functions with quality of life improvements
 License: MIT License
 Project-URL: Homepage, https://github.com/jessegrabowski/better_optimize
@@ -19,10 +19,10 @@ Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: numpy
-Requires-Dist: scipy>=1.15
+Requires-Dist: scipy>=1.18
 Requires-Dist: rich
 Requires-Dist: threadpoolctl
-Requires-Dist: joblib
+Requires-Dist: joblib>=1.4
 Requires-Dist: pandas
 Provides-Extra: tests
 Requires-Dist: pytest; extra == "tests"
@@ -38,6 +38,7 @@ include:
 - Progress bar!
 - Early stopping!
 - Better propagation of common arguments (`maxiters`, `tol`)!
+- A typed, documented configuration object for every method, so swapping optimizers is a one-argument change!
 
 ## Installation
 
@@ -68,39 +69,49 @@ All optimization routines in `better_optimize` can display a rich, informative p
 
 - No more nested `options` dictionaries! You can pass `tol`, `maxiter`, and other common options directly as top-level keyword arguments.
 - `better_optimize` automatically sorts and promotes these arguments to the correct place for each optimizer.
-- Generalizes argument handling: always provides `tol` and `maxiter` (or their equivalents) to the optimizer, even if you forget.
+- Generalizes argument handling: always provides `tol` and `maxiter` (or their equivalents) to the optimizer, even if you forget. `root` methods disagree about whether that budget is called `maxiter`, `maxfev` or `maxfun`, and you can pass `maxiter` to any of them.
+- Seven of the ten `root` methods take part of their configuration nested inside `jac_options`, which scipy splats into the jacobian it builds. Pass it as a mapping or as the typed object for that jacobian (`BroydenJacOptions`, `KrylovJacOptions`, and so on). An option that belongs there raises if you pass it at the top level, where scipy would warn and drop it.
 
-### 3. Argument Checking and Validation
+### 3. Typed Configuration Objects
+
+Every method scipy supports has a dataclass whose fields are exactly the options that method accepts, typed, defaulted, and documented with scipy's own prose. `help(BFGSConfig)` tells you what `c2` does without a trip to `scipy.optimize.show_options`.
+
+- Pass one wherever a method name works: `minimize(f, x0, method=BFGSConfig(gtol=1e-8))`. The string form keeps working unchanged.
+- A misspelled option raises `TypeError` naming it, rather than reaching scipy and being dropped with a warning nobody reads.
+- Fields and defaults are checked against scipy's own function signatures by test, so a scipy release that moves a default fails a test instead of drifting quietly.
+- One `minimize` call reaches local minimizers, basinhopping and differential evolution alike, because the configuration carries which solver runs it.
+
+### 4. Argument Checking and Validation
 
 - Automatic checking of provided gradient (`jac`), Hessian (`hess`), and Hessian-vector (`hessp`) functions.
 - Warns if you provide unnecessary or unused arguments for a given method.
 - Detects and handles fused objective functions (e.g., functions returning `(loss, grad)` or `(loss, grad, hess)` tuples).
 - Ensures that the correct function signatures and return types are used for each optimizer.
 
-### 4. LRUCache1 for Fused Functions
+### 5. LRUCache1 for Fused Functions
 
 - Provides an `LRUCache1` utility to cache the results of expensive objective/gradient/Hessian computations.
 - Especially useful for triple-fused functions that return value, gradient, and Hessian together, avoiding redundant computation.
 - Totally invisible -- just pass a function with 3 return values. Seamlessly integrated into the optimization workflow.
 
-### 5. Robust Basin-Hopping with Failure Tolerance
+### 6. Robust Basin-Hopping with Failure Tolerance
 
 - Enhanced `basinhopping` implementation allows you to continue even if the local minimizer fails.
 - Optionally accepts and stores failed minimizer results if they improve the global minimum.
 - Useful for noisy or non-smooth objective functions where local minimization may occasionally fail.
 
-### 6. Uniform Callback API
+### 7. Uniform Callback API
 
 scipy passes callbacks a different argument for almost every method: `callback(xk)`, `callback(xk, state)`,
 `callback(intermediate_result)`, or `callback(x, f)` for root finding. `better_optimize` normalizes them to a single
-`callback(res)`, where `res` is an `OptimizeResult` with the current `res.x`, `res.fun`, and `res.nit` — plus `res.jac`
+`callback(res)`, where `res` is an `OptimizeResult` with the current `res.x`, `res.fun`, and `res.nit` -- plus `res.jac`
 when a gradient is available, and solver-specific fields like `res.accept` (basinhopping) or `res.convergence`
 (differential evolution).
 
 - The return value is ignored, so a callback can report a value (an ELBO, a logged loss) without stopping the run.
 - Raise `StopOptimization` to stop early; the best result so far is returned with `success=False`.
 - The same callback works across `minimize`, `root`, `basinhopping`, and `differential_evolution`. `hybr` and `lm` are
-  the exception — scipy never calls a callback for them, so it is ignored with a warning.
+  the exception -- scipy never calls a callback for them, so it is ignored with a warning.
 
 ---
 
@@ -111,6 +122,7 @@ when a gradient is available, and solver-specific fields like `res.accept` (basi
 ```python
 import numpy as np
 from better_optimize import minimize
+from better_optimize.configuration import LBFGSBConfig
 
 def rosenbrock(x):
     return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
@@ -118,9 +130,7 @@ def rosenbrock(x):
 result = minimize(
     rosenbrock,
     x0=np.array([-1.0, 2.0]),
-    method="L-BFGS-B",
-    tol=1e-6,
-    maxiter=1000,
+    method=LBFGSBConfig(tol=1e-6, maxiter=1000),
     progressbar=True,  # Show a rich progress bar!
 )
 ```
@@ -133,6 +143,78 @@ result = minimize(
 
 The result object is a standard `OptimizeResult` from `scipy.optimize`, so there are no surprises there!
 
+### Swapping Optimizers
+
+A configuration object says which solver runs it, so trying a different optimizer family is a change to one argument rather than a rewrite. Nothing below changes but the loop variable.
+
+```python
+import numpy as np
+from better_optimize import minimize
+from better_optimize.configuration import (
+    BasinHoppingConfig,
+    DifferentialEvolutionConfig,
+    LBFGSBConfig,
+)
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+x0 = np.array([-1.0, 2.0])
+bounds = [(-5.0, 5.0), (-5.0, 5.0)]
+
+for config in [
+    LBFGSBConfig(maxiter=500),
+    BasinHoppingConfig(niter=25, minimizer_config=LBFGSBConfig()),
+    DifferentialEvolutionConfig(popsize=20, rng=0),
+]:
+    result = minimize(rosenbrock, x0, method=config, bounds=bounds, progressbar=False)
+    print(f"{config.method_name:25s} {result.fun:.3e}")
+```
+
+```shell
+L-BFGS-B                  9.191e-12
+basinhopping              3.696e-12
+differential_evolution    4.980e-30
+```
+
+`L-BFGS-B` is a local minimizer, `basinhopping` restarts one from perturbed points, and `differential_evolution` is a population method that never calls the local minimizer at all. They take different arguments, and scipy reaches them through different entry points. The configuration object is what absorbs that difference.
+
+### scipy Backwards Compatibility
+
+For users who have already memorized every combination of algorithm and option, none of the above is required. Method names still work, and so do flat keywords -- including the ones scipy keeps inside its `options` dictionary, which `better_optimize` gathers up for you. If you memorized that part too, pass `options` yourself and it is merged and checked the same way.
+
+```python
+import numpy as np
+from better_optimize import minimize
+from better_optimize.configuration import LBFGSBConfig
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+x0 = np.array([-1.0, 2.0])
+
+# A configuration object
+minimize(rosenbrock, x0, method=LBFGSBConfig(ftol=1e-6, gtol=1e-6, maxiter=1000, maxfun=1000))
+
+# Flat keywords, including ftol and gtol, which scipy keeps in `options`
+minimize(rosenbrock, x0, method="L-BFGS-B", ftol=1e-6, gtol=1e-6, maxiter=1000)
+
+# Or the options dictionary itself
+minimize(rosenbrock, x0, method="L-BFGS-B",
+         options={"ftol": 1e-6, "gtol": 1e-6, "maxiter": 1000, "maxfun": 1000})
+```
+
+Those three are the same call and return the same answer. The flat form additionally spreads a top-level `maxiter` across every budget the method has, which for `L-BFGS-B` means `maxfun` -- that is why the first and third forms name it and the second does not.
+
+All three are checked the same way, which is the part that is not backwards compatible. A misspelled option raises `TypeError` before the run starts, with a suggestion:
+
+```python
+minimize(rosenbrock, x0, method="L-BFGS-B", options={"gtoll": 1e-6})
+# TypeError: LBFGSBConfig.__init__() got an unexpected keyword argument 'gtoll'. Did you mean 'gtol'?
+```
+
+scipy would have accepted that call, run to completion, and mentioned the dropped option in an `OptimizeWarning` afterwards.
+
 ### Callbacks and Early Stopping
 
 Pass a `callback` and it runs after each iteration with an `OptimizeResult`. Its return value is ignored; raise
@@ -141,6 +223,7 @@ Pass a `callback` and it runs after each iteration with an `OptimizeResult`. Its
 ```python
 import numpy as np
 from better_optimize import minimize, StopOptimization
+from better_optimize.configuration import LBFGSBConfig
 
 def rosenbrock(x):
     return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
@@ -155,7 +238,7 @@ def callback(res):
 result = minimize(
     rosenbrock,
     x0=np.array([-1.0, 2.0]),
-    method="L-BFGS-B",
+    method=LBFGSBConfig(),
     callback=callback,
 )
 ```
@@ -163,10 +246,32 @@ result = minimize(
 The same callback works for `root` (`res.fun` is the residual vector), `basinhopping` (`res.accept`), and
 `differential_evolution` (`res.convergence`).
 
+### Interrupting a Run
+
+Ctrl-C raises `OptimizationInterrupted`, a subclass of `KeyboardInterrupt`, so it stops a loop that calls a solver.
+The partial result is on its `result` attribute, of the same type the solver returns: the last finite iterate for
+`minimize` and `root`, the best basin for `basinhopping`, the best point evaluated so far for `differential_evolution`,
+the finished stages for `sequential_optimize`, and the finished runs for `multi_optimize`. For the nested solvers, the
+inner solver's partial result is on `__cause__.result`. The objective is not called again after the interrupt, so a
+stalled objective does not stall the cleanup.
+
+```python
+from better_optimize import minimize, OptimizationInterrupted
+
+try:
+    result = minimize(rosenbrock, x0=np.array([-1.0, 2.0]), method="L-BFGS-B")
+except OptimizationInterrupted as interrupt:
+    result = interrupt.result
+```
+
+In IPython or Jupyter the `try` is unnecessary: after the traceback, the partial result is at
+`sys.last_value.result`.
+
 ### Triple-Fused Function using Pytensor
 
 ```python
 from better_optimize import minimize
+from better_optimize.configuration import NewtonCGConfig
 import pytensor.tensor as pt
 from pytensor import function
 import numpy as np
@@ -182,9 +287,7 @@ x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
 result = minimize(
     fused_fn, # No need to set flags separately, `better_optimize` handles it!
     x0=x0,
-    method="Newton-CG",
-    tol=1e-6,
-    maxiter=1000,
+    method=NewtonCGConfig(tol=1e-6, maxiter=1000),
     progressbar=True,  # Show a rich progress bar!
 )
 ```
@@ -193,6 +296,55 @@ Many sub-computations are repeated between the objective, gradient, and hessian
 fused value_and_grad function, but `better_optimize` also lets you pass a triple-fused value_grad_and_hess function.
 This avoids redundant computation and speeds up the optimization process.
 
+### Global Optimization with Differential Evolution
+
+`differential_evolution` searches a bounded region with a population instead of descending from a starting point, which suits objectives with many local minima and no useful gradient. It takes `bounds` rather than `x0`, and gets the same progress bar and callback treatment as everything else.
+
+```python
+import numpy as np
+from better_optimize import differential_evolution
+
+def rastrigin(x):
+    return 10 * len(x) + sum(x**2 - 10 * np.cos(2 * np.pi * x))
+
+result = differential_evolution(
+    rastrigin,
+    bounds=[(-5.12, 5.12)] * 5,
+    popsize=20,
+    rng=0,
+)
+
+print(result.x)    # [-0. -0. -0. -0. -0.]
+print(result.fun)  # 0.0
+```
+
+Rastrigin has a local minimum at roughly every integer lattice point, so a gradient method started anywhere but the middle converges to the wrong one.
+
+### Chaining Optimizers with `sequential_optimize`
+
+It is often quicker to get close with a robust method and finish with a fast one. `sequential_optimize` runs a list of stages, forwarding each stage's best point into the next stage's starting point, and reports what every stage did.
+
+```python
+import numpy as np
+from better_optimize import sequential_optimize
+from better_optimize.configuration import BFGSConfig, NelderMeadConfig
+
+def rosenbrock(x):
+    return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
+
+result = sequential_optimize(
+    rosenbrock,
+    x0=np.array([-1.2, 1.0]),
+    stages=[NelderMeadConfig(maxiter=200), BFGSConfig(gtol=1e-10)],
+)
+
+print(result.x)           # [0.99999552 0.99999104]
+print(result.best_stage)  # 1
+result.summary()          # Rich table of every stage, with the winner starred
+```
+
+A stage can also be a dict carrying its own `solver` and keyword arguments, which is what you need when a stage wants its own `bounds`, `constraints`, or starting point. By default the chain stops at the first stage that fails. Pass `on_failure="continue"` to run the rest anyway and keep the best result from those that worked.
+
 ### Parallel Optimization from Multiple Starting Points
 
 Real-world objectives often have multiple local minima. A common workaround is to throw many random starting points at
@@ -201,13 +353,14 @@ the optimizer and keep the best result. `better_optimize` makes this painless wi
 ```python
 import numpy as np
 from better_optimize import minimize, multi_optimize
+from better_optimize.configuration import LBFGSBConfig
 
 def rosenbrock(x):
     return sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0)
 
 result = multi_optimize(
     solver=minimize,
-    solver_kwargs=dict(f=rosenbrock, method="L-BFGS-B", tol=1e-10),
+    solver_kwargs=dict(f=rosenbrock, method=LBFGSBConfig(tol=1e-10)),
     x0=np.zeros(5),
     n_runs=16,
     init_strategy="uniform",
@@ -227,18 +380,18 @@ result.summary()          # Rich table of all runs, ranked
 > The `loky` (process) backend needs an `if __name__ == "__main__":` guard when run as a script on macOS or Windows.
 > Notebooks and the `threading`/`sequential` backends don't.
 
-`multi_optimize` works with **any** solver that follows the `(x0, **kwargs) → OptimizeResult` signature — that includes
+`multi_optimize` works with **any** solver that follows the `(x0, **kwargs) -> OptimizeResult` signature -- that includes
 `minimize`, `root`, `basinhopping`, or your own custom wrapper. It just calls `solver(x0=x0_i, **solver_kwargs)` for
 each starting point; it never inspects the solver internals.
 
 A few highlights:
 
-- **Initialization strategies** — `"uniform"`, `"normal"`, `"sobol"`, `"lhs"`, or pass your own callable. Bounded
+- **Initialization strategies** -- `"uniform"`, `"normal"`, `"sobol"`, `"lhs"`, or pass your own callable. Bounded
   strategies (`uniform`, `sobol`, `lhs`) require a `bounds` argument; `"normal"` perturbs around `x0` with a
   configurable `init_scale`. Or just pass an explicit `list[np.ndarray]` as `x0` and skip the generation entirely.
-- **Parallel backends** — `"sequential"` (for debugging), `"loky"` (CPU-bound work, default), or `"threading"`
+- **Parallel backends** -- `"sequential"` (for debugging), `"loky"` (CPU-bound work, default), or `"threading"`
   (GIL-releasing code). Under the hood this is `joblib`, so the usual `n_jobs=-1` convention works.
-- **BLAS thread control** — When many workers each spawn a full BLAS/OpenMP thread pool, you get noisy-neighbor
+- **BLAS thread control** -- When many workers each spawn a full BLAS/OpenMP thread pool, you get noisy-neighbor
   over-subscription. The `blas_cores` argument (default `"auto"`) caps the total thread budget so workers don't
   fight over the same cores.