hone-python 0.9.0__tar.gz → 0.11.0__tar.gz

{hone_python-0.9.0 → hone_python-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hone-python
-Version: 0.9.0
+Version: 0.11.0
 Summary: A python-based optimization engine/library.
 License-Expression: BSD-3-Clause
 License-File: LICENSE
@@ -178,6 +178,78 @@ RangeParameter("n_estimators", 10, 1000, integer=True, log=True)
 
 Every parameter the objective function requires must be in the search space. The optimizer checks this before it starts.
 
+### Constraints
+
+Limits across parameters are written with the parameters themselves and passed to `SearchSpace`. Every optimizer then only asks for points that satisfy every constraint, so the objective is never evaluated outside them:
+
+```python
+from hone import RangeParameter, SearchSpace
+
+width = RangeParameter("width", 1, 10)
+height = RangeParameter("height", 1, 10)
+search_space = SearchSpace([width, height], constraints=[width * height <= 40, width >= 2 * height])
+```
+
+Range parameters and numbers combine with `+`, `-`, `*`, `/`, `**` and `abs()`, and `<=` or `>=` makes the result a constraint; `<` and `>` mean the same. A point where a constraint can't be computed, such as one that divides by zero, counts as breaking it. There are no equality constraints: an equality leaves nothing to sample, so remove a variable instead and compute it inside the objective. Write a range as two constraints, since Python reads `0 <= width - height <= 5` as two comparisons joined by `and`, and hone raises an error rather than keep one half.
+
+For anything an expression can't say, such as a square root or a choice parameter, use a function, which takes parameters by name and is satisfied when it returns zero or less:
+
+```python
+from hone import Constraint
+
+def stress(width, height, material):
+    return bending_stress(width, height) - ALLOWED[material]
+
+search_space = SearchSpace([width, height, material], constraints=[Constraint.function(stress)])
+```
+
+Constraints should be quick to check: hone checks them for every candidate it considers, which can be many times the number of evaluations. Expression constraints are saved with a result; a function constraint saves only its name, so after `Result.load` pass the function again with `rebuild_optimizer(functions={"stress": stress})`.
+
+Starting points must satisfy the constraints. If the constraints leave nothing to search, or a region too small to find in 100,000 random draws, the optimizer raises `InfeasibleSearchSpace` when it's created, before any evaluation. Each optimizer keeps to constraints in its own way: random search and Bayesian optimization only consider points that satisfy them, CMA-ES draws a candidate again while it breaks one, and the genetic optimizer moves such a child back toward its fitter parent until it is feasible, which tends to land it near the boundary, where constrained optima usually are. On the benchmark's constrained problems the choice between optimizers follows the same pattern as without constraints; see [benchmarks/README.md](benchmarks/README.md#constrained-problems). [examples/pressure_vessel.py](examples/pressure_vessel.py) designs a pressure vessel of least cost under three constraints and finds the best published design. [examples/maintenance_schedule.py](examples/maintenance_schedule.py) chooses a maintenance schedule for a production line, scored by a noisy simulation of a year, with a constraint on technician hours and a function constraint that depends on the chosen shift; it halves the expected yearly cost of a sensible schedule chosen by hand.
+
+### Conditional parameters
+
+Some settings only exist when another takes certain values: a kernel's settings only for the model that has kernels, a third layer's size only with three layers or more. Write each condition as `test >> parameters`, read "then", and pass them to `SearchSpace`:
+
+```python
+from hone import ChoiceParameter, RangeParameter, SearchSpace
+
+model = ChoiceParameter("model", ["svm", "forest"])
+kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+C = RangeParameter("C", 1e-3, 1e3, log=True)
+gamma = RangeParameter("gamma", 1e-5, 1, log=True)
+trees = RangeParameter("trees", 10, 500, integer=True)
+
+search_space = SearchSpace(
+    [model, kernel, C, gamma, trees],
+    conditions=[
+        model["svm"] >> [kernel, C],
+        kernel["rbf"] >> gamma,  # only with an svm too, since kernel is
+        model["forest"] >> trees,
+    ],
+)
+
+def train(model, kernel=None, C=None, gamma=None, trees=None):
+    ...
+```
+
+The left of `>>` is a test: a choice parameter indexed by the values that switch the parameters on, such as `model["svm"]` or `optimizer["sgd", "nesterov"]`, or a comparison of range parameters in brackets, such as `(layers >= 3) >> units_3`. The brackets are needed, since Python reads `layers >= 3 >> units_3` as `layers >= (3 >> units_3)`; hone raises an error saying so. The right is a parameter or a list of them, and each parameter can have one condition. Conditions nest: a parameter whose test uses a parameter that is switched off is off too.
+
+A parameter that is switched off isn't part of the point. The objective isn't passed it, so give it a default, which hone checks before the run starts; results, progress and `best_params` leave it out; and a starting point must leave it out too. Two points that differ only in a parameter switched off are the same point, so they are averaged together with `noise=True` and the genetic optimizer doesn't count one as new.
+
+A constraint applies only while every parameter it uses is switched on, and `test >> constraint` makes a constraint apply only while the test holds:
+
+```python
+depth = RangeParameter("depth", 2, 20, integer=True)
+search_space = SearchSpace(
+    [model, trees, depth],
+    constraints=[model["forest"] >> (trees * depth <= 2000)],
+    conditions=[model["forest"] >> [trees, depth]],
+)
+```
+
+Every optimizer supports conditions; CMA-ES only conditions on range parameters, since it doesn't search choices. Bayesian optimization gains the most: a parameter switched off looks the same to its model whatever value it holds, so values that don't matter add no noise. On the benchmark's conditional problems its error is ×0.0004 that of searching every parameter all the time, at 10 evaluations per dimension. With conditions the genetic optimizer breeds each branch within its own niche by default (`niche_share` of 0.5, below), so a wide, easy branch doesn't crowd out a narrow, better one; see [benchmarks/README.md](benchmarks/README.md#conditional-problems). [examples/choose_a_model.py](examples/choose_a_model.py) chooses a classifier, among an SVM with three kernels, a random forest and logistic regression, together with its settings.
+
 ### Configuration
 
 `GeneticOptimizerConfig` validates its values when created. It can also be loaded from a JSON or YAML file:
@@ -190,7 +262,7 @@ Set `seed` to make a run repeatable. `max_evaluations` sets a budget: the run st
 
 Parents are chosen by tournament: `tournament_size` (default 4) individuals are drawn and the fittest wins.
 
-When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default.
+When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default, except in a search space with conditions: there each branch is a niche, and 0.5 is the default, since a branch with a better optimum is often harder to find.
 
 Each value of a new individual mutates with probability `mutation_rate`: by default 1 over the number of variables, and at least 0.1, so in two variables half of them mutate. A range value moves by a random step whose typical size is a fraction of the width of its range. A choice switches to another option, and a fixed value never changes.
 
@@ -326,6 +398,7 @@ Requires Python 3.12 or later and [Poetry](https://python-poetry.org/) 2.
 | `make benchmark` | Run the full benchmark suite and update `benchmarks/results.md` |
 | `make benchmark-noisy` | Run the noisy problems and update `benchmarks/noisy-results.md` |
 | `make benchmark-ml` | Run the machine learning tuning problems and update `benchmarks/ml-results.md` |
+| `make benchmark-constrained` | Run the constrained problems and update `benchmarks/constrained-results.md` |
 
 To release, raise `version` in `pyproject.toml` and add a section for it to `CHANGELOG.md`. Once the change is on `main` and its checks pass, CI tags the version, publishes a GitHub release with that section as its notes and the built package attached, and uploads the same package to PyPI as `hone-python`.
 

{hone_python-0.9.0 → hone_python-0.11.0}/README.md

@@ -157,6 +157,78 @@ RangeParameter("n_estimators", 10, 1000, integer=True, log=True)
 
 Every parameter the objective function requires must be in the search space. The optimizer checks this before it starts.
 
+### Constraints
+
+Limits across parameters are written with the parameters themselves and passed to `SearchSpace`. Every optimizer then only asks for points that satisfy every constraint, so the objective is never evaluated outside them:
+
+```python
+from hone import RangeParameter, SearchSpace
+
+width = RangeParameter("width", 1, 10)
+height = RangeParameter("height", 1, 10)
+search_space = SearchSpace([width, height], constraints=[width * height <= 40, width >= 2 * height])
+```
+
+Range parameters and numbers combine with `+`, `-`, `*`, `/`, `**` and `abs()`, and `<=` or `>=` makes the result a constraint; `<` and `>` mean the same. A point where a constraint can't be computed, such as one that divides by zero, counts as breaking it. There are no equality constraints: an equality leaves nothing to sample, so remove a variable instead and compute it inside the objective. Write a range as two constraints, since Python reads `0 <= width - height <= 5` as two comparisons joined by `and`, and hone raises an error rather than keep one half.
+
+For anything an expression can't say, such as a square root or a choice parameter, use a function, which takes parameters by name and is satisfied when it returns zero or less:
+
+```python
+from hone import Constraint
+
+def stress(width, height, material):
+    return bending_stress(width, height) - ALLOWED[material]
+
+search_space = SearchSpace([width, height, material], constraints=[Constraint.function(stress)])
+```
+
+Constraints should be quick to check: hone checks them for every candidate it considers, which can be many times the number of evaluations. Expression constraints are saved with a result; a function constraint saves only its name, so after `Result.load` pass the function again with `rebuild_optimizer(functions={"stress": stress})`.
+
+Starting points must satisfy the constraints. If the constraints leave nothing to search, or a region too small to find in 100,000 random draws, the optimizer raises `InfeasibleSearchSpace` when it's created, before any evaluation. Each optimizer keeps to constraints in its own way: random search and Bayesian optimization only consider points that satisfy them, CMA-ES draws a candidate again while it breaks one, and the genetic optimizer moves such a child back toward its fitter parent until it is feasible, which tends to land it near the boundary, where constrained optima usually are. On the benchmark's constrained problems the choice between optimizers follows the same pattern as without constraints; see [benchmarks/README.md](benchmarks/README.md#constrained-problems). [examples/pressure_vessel.py](examples/pressure_vessel.py) designs a pressure vessel of least cost under three constraints and finds the best published design. [examples/maintenance_schedule.py](examples/maintenance_schedule.py) chooses a maintenance schedule for a production line, scored by a noisy simulation of a year, with a constraint on technician hours and a function constraint that depends on the chosen shift; it halves the expected yearly cost of a sensible schedule chosen by hand.
+
+### Conditional parameters
+
+Some settings only exist when another takes certain values: a kernel's settings only for the model that has kernels, a third layer's size only with three layers or more. Write each condition as `test >> parameters`, read "then", and pass them to `SearchSpace`:
+
+```python
+from hone import ChoiceParameter, RangeParameter, SearchSpace
+
+model = ChoiceParameter("model", ["svm", "forest"])
+kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+C = RangeParameter("C", 1e-3, 1e3, log=True)
+gamma = RangeParameter("gamma", 1e-5, 1, log=True)
+trees = RangeParameter("trees", 10, 500, integer=True)
+
+search_space = SearchSpace(
+    [model, kernel, C, gamma, trees],
+    conditions=[
+        model["svm"] >> [kernel, C],
+        kernel["rbf"] >> gamma,  # only with an svm too, since kernel is
+        model["forest"] >> trees,
+    ],
+)
+
+def train(model, kernel=None, C=None, gamma=None, trees=None):
+    ...
+```
+
+The left of `>>` is a test: a choice parameter indexed by the values that switch the parameters on, such as `model["svm"]` or `optimizer["sgd", "nesterov"]`, or a comparison of range parameters in brackets, such as `(layers >= 3) >> units_3`. The brackets are needed, since Python reads `layers >= 3 >> units_3` as `layers >= (3 >> units_3)`; hone raises an error saying so. The right is a parameter or a list of them, and each parameter can have one condition. Conditions nest: a parameter whose test uses a parameter that is switched off is off too.
+
+A parameter that is switched off isn't part of the point. The objective isn't passed it, so give it a default, which hone checks before the run starts; results, progress and `best_params` leave it out; and a starting point must leave it out too. Two points that differ only in a parameter switched off are the same point, so they are averaged together with `noise=True` and the genetic optimizer doesn't count one as new.
+
+A constraint applies only while every parameter it uses is switched on, and `test >> constraint` makes a constraint apply only while the test holds:
+
+```python
+depth = RangeParameter("depth", 2, 20, integer=True)
+search_space = SearchSpace(
+    [model, trees, depth],
+    constraints=[model["forest"] >> (trees * depth <= 2000)],
+    conditions=[model["forest"] >> [trees, depth]],
+)
+```
+
+Every optimizer supports conditions; CMA-ES only conditions on range parameters, since it doesn't search choices. Bayesian optimization gains the most: a parameter switched off looks the same to its model whatever value it holds, so values that don't matter add no noise. On the benchmark's conditional problems its error is ×0.0004 that of searching every parameter all the time, at 10 evaluations per dimension. With conditions the genetic optimizer breeds each branch within its own niche by default (`niche_share` of 0.5, below), so a wide, easy branch doesn't crowd out a narrow, better one; see [benchmarks/README.md](benchmarks/README.md#conditional-problems). [examples/choose_a_model.py](examples/choose_a_model.py) chooses a classifier, among an SVM with three kernels, a random forest and logistic regression, together with its settings.
+
 ### Configuration
 
 `GeneticOptimizerConfig` validates its values when created. It can also be loaded from a JSON or YAML file:
@@ -169,7 +241,7 @@ Set `seed` to make a run repeatable. `max_evaluations` sets a budget: the run st
 
 Parents are chosen by tournament: `tournament_size` (default 4) individuals are drawn and the fittest wins.
 
-When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default.
+When a choice parameter's best option is found late, the genetic optimizer can settle on another one: once most of the population uses one option, a child with a different option inherits numbers tuned for the wrong one and loses. `niche_share=0.5` breeds half of each generation's children within niches of parents that share their choices, so every option keeps tuning its own numbers. On material-selection-10d with 1,000 evaluations per dimension it found the right material in 10 of 10 runs, against 6 of 10 without. It costs some progress on small budgets, so it's off by default, except in a search space with conditions: there each branch is a niche, and 0.5 is the default, since a branch with a better optimum is often harder to find.
 
 Each value of a new individual mutates with probability `mutation_rate`: by default 1 over the number of variables, and at least 0.1, so in two variables half of them mutate. A range value moves by a random step whose typical size is a fraction of the width of its range. A choice switches to another option, and a fixed value never changes.
 
@@ -305,6 +377,7 @@ Requires Python 3.12 or later and [Poetry](https://python-poetry.org/) 2.
 | `make benchmark` | Run the full benchmark suite and update `benchmarks/results.md` |
 | `make benchmark-noisy` | Run the noisy problems and update `benchmarks/noisy-results.md` |
 | `make benchmark-ml` | Run the machine learning tuning problems and update `benchmarks/ml-results.md` |
+| `make benchmark-constrained` | Run the constrained problems and update `benchmarks/constrained-results.md` |
 
 To release, raise `version` in `pyproject.toml` and add a section for it to `CHANGELOG.md`. Once the change is on `main` and its checks pass, CI tags the version, publishes a GitHub release with that section as its notes and the built package attached, and uploads the same package to PyPI as `hone-python`.
 

{hone_python-0.9.0 → hone_python-0.11.0}/hone/__init__.py

@@ -10,7 +10,7 @@ from hone.optimizers import (
     UnsupportedSearchSpace,
     choose_optimizer,
 )
-from hone.parameters import ChoiceParameter, FixedParameter, RangeParameter, SearchSpace
+from hone.parameters import ChoiceParameter, Condition, Constraint, FixedParameter, InfeasibleSearchSpace, RangeParameter, SearchSpace
 from hone.result import GenerationStats, Progress, Result, RunInfo, print_progress
 
 __all__ = [
@@ -19,11 +19,14 @@ __all__ = [
     "CMAES",
     "CMAESConfig",
     "ChoiceParameter",
+    "Condition",
+    "Constraint",
     "EvaluationFailed",
     "FixedParameter",
     "GenerationStats",
     "GeneticOptimizer",
     "GeneticOptimizerConfig",
+    "InfeasibleSearchSpace",
     "Optimizer",
     "OptimizerConfig",
     "Progress",

{hone_python-0.9.0 → hone_python-0.11.0}/hone/benchmarks/__init__.py

@@ -1,4 +1,6 @@
 """Benchmark problems and a runner for comparing optimizers. Run `python -m hone.benchmarks --help`."""
+from .conditional import conditional_problems
+from .constrained import constrained_problems
 from .ml import ml_problems
 from .problems import Problem, noisy_problems, quick_problems, standard_problems
 from .runner import (

{hone_python-0.9.0 → hone_python-0.11.0}/hone/benchmarks/__main__.py

@@ -6,6 +6,8 @@ from collections.abc import Sequence
 from importlib.metadata import version
 
 from . import ml
+from .conditional import conditional_problems
+from .constrained import constrained_problems
 from .problems import noisy_problems, quick_problems, standard_problems
 from .runner import BUDGETS, load_summaries, regressions, run_benchmark, save_summaries, summarize, to_markdown
 
@@ -26,6 +28,10 @@ def main(argv: list[str] | None = None) -> int:
     suite.add_argument("--quick", action="store_true", help="run the small CI subset with fewer seeds and evaluations")
     suite.add_argument("--ml", action="store_true", help="run the machine learning tuning problems instead (needs scikit-learn)")
     suite.add_argument("--noisy", action="store_true", help="run the problems with noise added to every evaluation instead")
+    suite.add_argument("--constrained", action="store_true", help="run the problems with constraints instead")
+    suite.add_argument(
+        "--conditional", action="store_true", help="run the problems with conditional parameters, and their flat versions, instead"
+    )
     parser.add_argument("--seeds", type=int, help="number of seeds per problem (default 10, or 5 with --quick)")
     parser.add_argument(
         "--budgets",
@@ -49,6 +55,10 @@ def main(argv: list[str] | None = None) -> int:
         problems, default_budgets = ml.ml_problems(), ml.BUDGETS
     elif args.noisy:
         problems, default_budgets = noisy_problems(), NOISY_BUDGETS
+    elif args.constrained:
+        problems, default_budgets = constrained_problems(), BUDGETS
+    elif args.conditional:
+        problems, default_budgets = conditional_problems(), BUDGETS
     else:
         problems, default_budgets = standard_problems(), BUDGETS
     seeds = args.seeds or (5 if args.quick else 10)

hone_python-0.11.0/hone/benchmarks/conditional.py

@@ -0,0 +1,109 @@
+"""Conditional test problems, each in two versions: with conditions, and with every parameter switched on.
+
+- branches-3x2d and branches-3x5d: a choice of three branches, each a function of its own
+  numbers. The best branch is a steep, narrow bowl reaching 0; the decoys are a wide, shallow
+  bowl whose floor is 1 and a shallow Rastrigin grid whose floor is 0.5. Random points score
+  better in the decoys, so an optimizer has to look past them.
+- nested-9: a choice of model; one model has a choice of kernel, and one kernel has numbers of
+  its own, so the best point is two conditions deep. The other model is a wide, shallow decoy
+  whose floor is 0.25.
+- layers-7: a layer count from 1 to 5 whose larger values switch on more numbers, as a
+  network's units per layer would be. Three layers is best. Its conditions test a range, so
+  CMA-ES can search it.
+
+The "-flat" version of each has the same parameters, function and optimum, with no conditions:
+every parameter is always on and the function ignores those it doesn't use, which was the way
+to search such a space before conditions. Comparing the two shows what conditions are worth.
+"""
+from typing import Any, Callable
+
+import numpy as np
+
+from hone.parameters import ChoiceParameter, Condition, Parameter, RangeParameter, SearchSpace
+
+from .problems import BOUND, Problem, _shift, rastrigin, sphere
+
+
+def _problem(name: str, function: Callable[..., float], parameters: list[Parameter], conditions: list[Condition]) -> list[Problem]:
+    function.__name__ = name.replace("-", "_")
+    return [
+        Problem(
+            name=name,
+            function=function,
+            search_space=SearchSpace(parameters, conditions=conditions),
+            optimum=0.0,
+            tags=frozenset({"conditional"}),
+        ),
+        Problem(
+            name=f"{name}-flat",
+            function=function,
+            search_space=SearchSpace(parameters),
+            optimum=0.0,
+            tags=frozenset({"conditional", "flat"}),
+        ),
+    ]
+
+
+def _numbers(prefix: str, count: int) -> list[RangeParameter]:
+    return [RangeParameter(f"{prefix}{i}", -BOUND, BOUND) for i in range(count)]
+
+
+def _vector(params: dict[str, Any], numbers: list[RangeParameter]) -> np.ndarray:
+    return np.array([params[p.name] for p in numbers], dtype=float)
+
+
+def branches(dimensions: int) -> list[Problem]:
+    name = f"branches-3x{dimensions}d"
+    branch = ChoiceParameter("branch", ["narrow", "wide", "grid"])
+    numbers = {b: _numbers(f"{b}_", dimensions) for b in branch.choices}
+    shifts = {b: _shift(f"{name}/{b}", dimensions) for b in branch.choices}
+    # (function, scale, floor) for each branch
+    bases = {"narrow": (sphere, 4.0, 0.0), "wide": (sphere, 0.05, 1.0), "grid": (rastrigin, 0.1, 0.5)}
+
+    def function(branch: str, **params: Any) -> float:
+        base, scale, floor = bases[branch]
+        return floor + scale * float(base(_vector(params, numbers[branch]) - shifts[branch]))
+
+    parameters: list[Parameter] = [branch, *(p for b in branch.choices for p in numbers[b])]
+    return _problem(name, function, parameters, [branch[b] >> numbers[b] for b in branch.choices])
+
+
+def nested() -> list[Problem]:
+    name = "nested-9"
+    model = ChoiceParameter("model", ["kernel_machine", "forest"])
+    kernel = ChoiceParameter("kernel", ["linear", "rbf"])
+    shared, width, forest = _numbers("s", 2), _numbers("g", 2), _numbers("f", 3)
+    centres = {key: _shift(f"{name}/{key}", len(group)) for key, group in [("s", shared), ("g", width), ("f", forest)]}
+
+    def function(model: str, kernel: str | None = None, **params: Any) -> float:
+        if model == "forest":
+            return 0.25 + 0.05 * float(sphere(_vector(params, forest) - centres["f"]))
+        value = float(sphere(_vector(params, shared) - centres["s"]))
+        if kernel == "linear":
+            return 0.5 + value
+        return value + 4 * float(sphere(_vector(params, width) - centres["g"]))
+
+    parameters: list[Parameter] = [model, kernel, *shared, *width, *forest]
+    conditions = [model["kernel_machine"] >> [kernel, *shared], kernel["rbf"] >> width, model["forest"] >> forest]
+    return _problem(name, function, parameters, conditions)
+
+
+def layers() -> list[Problem]:
+    name = "layers-7"
+    count = RangeParameter("layers", 1, 5, integer=True)
+    rate = RangeParameter("rate", -BOUND, BOUND)
+    units = _numbers("units_", 5)
+    centre = _shift(f"{name}/units", 5)
+    best_rate = float(_shift(f"{name}/rate", 1)[0])
+
+    def function(layers: int, rate: float, **params: Any) -> float:
+        used = units[:layers]
+        return 0.5 * abs(layers - 3) + float(sphere(_vector(params, used) - centre[:layers])) + (rate - best_rate) ** 2
+
+    parameters: list[Parameter] = [count, rate, *units]
+    return _problem(name, function, parameters, [(count >= i + 1) >> units[i] for i in range(1, 5)])
+
+
+def conditional_problems() -> list[Problem]:
+    """Each conditional problem, followed by its flat version."""
+    return [*branches(2), *branches(5), *nested(), *layers()]

hone_python-0.11.0/hone/benchmarks/constrained.py

@@ -0,0 +1,172 @@
+"""Constrained test problems, whose optima lie on a constraint's boundary, as they usually do in engineering.
+
+- boundary-sphere and boundary-ellipsoid: a shifted sphere or ellipsoid with a linear constraint
+  that cuts off its unconstrained optimum, so the answer is on the cutting plane. Both optima are
+  known exactly.
+- rosenbrock-disk: Rosenbrock's function inside the disk x² + y² ≤ 2, whose optimum (1, 1) sits
+  on the disk's edge.
+- pressure-vessel, tension-spring and welded-beam: three standard engineering design problems,
+  scored against the best designs published for them. The welded beam's stress formulas need
+  square roots, so its constraints are functions; the others are expressions.
+"""
+import math
+import zlib
+from typing import Any
+
+import numpy as np
+
+from hone.parameters import Constraint, RangeParameter, SearchSpace
+
+from .problems import BOUND, Problem, _shift
+
+
+def _normal(name: str, dimensions: int) -> np.ndarray:
+    """A fixed unit vector for the named problem, with no component near zero."""
+    rng = np.random.default_rng(zlib.crc32(f"{name}/normal".encode()))
+    w = rng.uniform(0.3, 1.0, dimensions) * rng.choice([-1.0, 1.0], dimensions)
+    return w / np.linalg.norm(w)
+
+
+def _boundary(name: str, dimensions: int, weights: np.ndarray) -> Problem:
+    """Minimize sum(weights * (x - c)²) subject to w·x ≤ w·c - 1, which the unconstrained optimum c breaks.
+
+    The optimum is at c + z with z = -D⁻¹w / (wᵀD⁻¹w), where D is diag(weights), and its value is 1 / (wᵀD⁻¹w).
+    """
+    names = [f"x{i}" for i in range(dimensions)]
+    parameters = [RangeParameter(n, -BOUND, BOUND) for n in names]
+    centre, w = _shift(name, dimensions), _normal(name, dimensions)
+    limit = float(w @ centre) - 1.0
+    terms = [float(wi) * p for wi, p in zip(w, parameters)]
+    plane = terms[0]
+    for term in terms[1:]:
+        plane = plane + term
+    inverse = 1.0 / weights
+    best = centre - inverse * w / float(w @ (inverse * w))
+    assert np.all(np.abs(best) <= BOUND), f"{name}: the constrained optimum must lie inside the bounds"
+
+    def function(**params: Any) -> float:
+        x = np.array([params[n] for n in names], dtype=float)
+        return float(np.sum(weights * (x - centre) ** 2))
+
+    function.__name__ = name.replace("-", "_")
+    space = SearchSpace(parameters, [plane <= limit])
+    return Problem(name=name, function=function, search_space=space, optimum=1.0 / float(w @ (inverse * w)), tags=frozenset({"constrained"}))
+
+
+def boundary_sphere(dimensions: int) -> Problem:
+    return _boundary(f"boundary-sphere-{dimensions}d", dimensions, np.ones(dimensions))
+
+
+def boundary_ellipsoid(dimensions: int) -> Problem:
+    # Axis weights from 1 to 1,000: wide enough to need scaling, narrow enough to keep the optimum in bounds.
+    return _boundary(f"boundary-ellipsoid-{dimensions}d", dimensions, 10.0 ** (3 * np.arange(dimensions) / max(dimensions - 1, 1)))
+
+
+def rosenbrock_disk() -> Problem:
+    x, y = RangeParameter("x", -1.5, 1.5), RangeParameter("y", -1.5, 1.5)
+
+    def rosenbrock_disk(x: float, y: float) -> float:
+        return (1 - x) ** 2 + 100 * (y - x**2) ** 2
+
+    space = SearchSpace([x, y], [x**2 + y**2 <= 2])
+    return Problem(name="rosenbrock-disk-2d", function=rosenbrock_disk, search_space=space, optimum=0.0, tags=frozenset({"constrained"}))
+
+
+def pressure_vessel() -> Problem:
+    """A cylindrical vessel with hemispherical heads, at least 1,296,000 cubic inches, of least cost.
+
+    Shell and head thicknesses come in sixteenths of an inch, so they are integers here. The best
+    published design costs 6059.714 (Ts = 13/16, Th = 7/16, R = 42.098, L = 176.637).
+    """
+    shell, head = RangeParameter("shell", 1, 99, integer=True), RangeParameter("head", 1, 99, integer=True)
+    radius, length = RangeParameter("radius", 10, 200), RangeParameter("length", 10, 200)
+
+    def pressure_vessel(shell: int, head: int, radius: float, length: float) -> float:
+        ts, th = shell / 16, head / 16
+        return 0.6224 * ts * radius * length + 1.7781 * th * radius**2 + 3.1661 * ts**2 * length + 19.84 * ts**2 * radius
+
+    space = SearchSpace(
+        [shell, head, radius, length],
+        [
+            shell / 16 >= 0.0193 * radius,
+            head / 16 >= 0.00954 * radius,
+            math.pi * radius**2 * length + 4 / 3 * math.pi * radius**3 >= 1_296_000,
+        ],
+    )
+    return Problem(name="pressure-vessel", function=pressure_vessel, search_space=space, optimum=6059.714335, tags=frozenset({"constrained", "mixed", "engineering"}))
+
+
+def tension_spring() -> Problem:
+    """A coil spring of least weight under limits on deflection, shear stress, surge frequency and diameter.
+
+    The best published design weighs 0.0126652 (d = 0.051690, D = 0.356750, N = 11.287126).
+    """
+    d, coil, n = RangeParameter("wire", 0.05, 2.0), RangeParameter("coil", 0.25, 1.3), RangeParameter("turns", 2.0, 15.0)
+
+    def tension_spring(wire: float, coil: float, turns: float) -> float:
+        return (turns + 2) * coil * wire**2
+
+    space = SearchSpace(
+        [d, coil, n],
+        [
+            coil**3 * n / (71785 * d**4) >= 1,
+            (4 * coil**2 - d * coil) / (12566 * (coil * d**3 - d**4)) + 1 / (5108 * d**2) <= 1,
+            140.45 * d / (coil**2 * n) >= 1,
+            d + coil <= 1.5,
+        ],
+    )
+    return Problem(name="tension-spring", function=tension_spring, search_space=space, optimum=0.012665233, tags=frozenset({"constrained", "engineering"}))
+
+
+# The welded beam's load and material: load P (lb), length L (in), Young's and shear moduli (psi).
+P, L, E, G = 6000.0, 14.0, 30e6, 12e6
+
+
+def _shear(weld: float, length: float, thickness: float) -> float:
+    primary = P / (math.sqrt(2) * weld * length)
+    moment = P * (L + length / 2)
+    r = math.sqrt(length**2 / 4 + ((weld + thickness) / 2) ** 2)
+    j = 2 * math.sqrt(2) * weld * length * (length**2 / 12 + ((weld + thickness) / 2) ** 2)
+    secondary = moment * r / j
+    return math.sqrt(primary**2 + primary * secondary * length / r + secondary**2)
+
+
+def _buckling(thickness: float, width: float) -> float:
+    return 4.013 * E * math.sqrt(thickness**2 * width**6 / 36) / L**2 * (1 - thickness / (2 * L) * math.sqrt(E / (4 * G)))
+
+
+def welded_beam() -> Problem:
+    """A beam welded to a support, of least cost under limits on shear, bending, buckling and deflection.
+
+    The best published design costs 1.724852 (h = 0.205730, l = 3.470489, t = 9.036624, b = 0.205730).
+    """
+    weld, length = RangeParameter("weld", 0.1, 2.0), RangeParameter("length", 0.1, 10.0)
+    thickness, width = RangeParameter("thickness", 0.1, 10.0), RangeParameter("width", 0.1, 2.0)
+
+    def welded_beam(weld: float, length: float, thickness: float, width: float) -> float:
+        return 1.10471 * weld**2 * length + 0.04811 * thickness * width * (14 + length)
+
+    def shear(weld: float, length: float, thickness: float) -> float:
+        return _shear(weld, length, thickness) - 13600
+
+    def buckling(thickness: float, width: float) -> float:
+        return P - _buckling(thickness, width)
+
+    space = SearchSpace(
+        [weld, length, thickness, width],
+        [
+            Constraint.function(shear),
+            6 * P * L / (width * thickness**2) <= 30000,
+            weld <= width,
+            0.10471 * weld**2 + 0.04811 * thickness * width * (14 + length) <= 5,
+            4 * P * L**3 / (E * thickness**3 * width) <= 0.25,
+            Constraint.function(buckling),
+        ],
+    )
+    return Problem(name="welded-beam", function=welded_beam, search_space=space, optimum=1.724852, tags=frozenset({"constrained", "engineering"}))
+
+
+def constrained_problems(dimensions: tuple[int, ...] = (2, 5, 10)) -> list[Problem]:
+    """The constrained suite: boundary sphere and ellipsoid in each dimension, Rosenbrock in a disk, and three engineering designs."""
+    problems = [factory(d) for d in dimensions for factory in (boundary_sphere, boundary_ellipsoid)]
+    return problems + [rosenbrock_disk(), pressure_vessel(), tension_spring(), welded_beam()]

{hone_python-0.9.0 → hone_python-0.11.0}/hone/benchmarks/ml.py

@@ -16,7 +16,7 @@ from typing import Callable
 
 from threadpoolctl import threadpool_limits
 
-from hone.parameters import ChoiceParameter, RangeParameter, SearchSpace
+from hone.parameters import ChoiceParameter, Condition, Parameter, RangeParameter, SearchSpace
 
 from .problems import Problem
 
@@ -92,11 +92,56 @@ def random_forest_wine(max_depth: int, min_samples_leaf: int, max_features: floa
     return _cross_validate(model, "wine", classification=True)
 
 
+def model_selection_breast_cancer(
+    model: str,
+    C: float | None = None,
+    gamma: float | None = None,
+    max_depth: int | None = None,
+    min_samples_leaf: int | None = None,
+    max_features: float | None = None,
+    n_neighbors: int | None = None,
+    weights: str | None = None,
+) -> float:
+    """One of three classifiers, and its settings, on the 569 breast tumour samples svm-breast-cancer uses.
+
+    A support vector classifier with an RBF kernel, a 20-tree random forest, or k nearest
+    neighbours. With conditions, only the chosen model's settings are searched.
+    """
+    from sklearn.ensemble import RandomForestClassifier
+    from sklearn.neighbors import KNeighborsClassifier
+    from sklearn.pipeline import make_pipeline
+    from sklearn.preprocessing import StandardScaler
+    from sklearn.svm import SVC
+
+    if model == "svm":
+        estimator = make_pipeline(StandardScaler(), SVC(C=C, gamma=gamma))
+    elif model == "forest":
+        estimator = RandomForestClassifier(
+            n_estimators=20, max_depth=max_depth, min_samples_leaf=min_samples_leaf, max_features=max_features, random_state=0, n_jobs=1
+        )
+    else:
+        estimator = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=n_neighbors, weights=weights))
+    return _cross_validate(estimator, "breast_cancer", classification=True)
+
+
+_MODEL = ChoiceParameter("model", ["svm", "forest", "knn"])
+_SVM = [RangeParameter("C", 1e-3, 1e3, log=True), RangeParameter("gamma", 1e-5, 10, log=True)]
+_FOREST = [
+    RangeParameter("max_depth", 1, 12, integer=True),
+    RangeParameter("min_samples_leaf", 1, 20, integer=True),
+    RangeParameter("max_features", 0.05, 1.0),
+]
+_KNN: list[Parameter] = [RangeParameter("n_neighbors", 1, 50, integer=True), ChoiceParameter("weights", ["uniform", "distance"])]
+MODEL_SELECTION: list[Parameter] = [_MODEL, *_SVM, *_FOREST, *_KNN]
+MODEL_CONDITIONS: list[Condition] = [_MODEL["svm"] >> _SVM, _MODEL["forest"] >> _FOREST, _MODEL["knn"] >> _KNN]
+
 # The best value of each problem found by REFERENCE_SEARCH.
 OPTIMA = {
     "svm-breast-cancer": 0.014066138798323302,  # accuracy 98.6%, found by CMA-ES
     "kernel-ridge-diabetes": 0.4961748264794844,  # R² 0.504, found by Bayesian optimization
     "random-forest-wine": 0.005555555555555536,  # accuracy 99.4%, found by Bayesian optimization
+    # Every reference run chose the support vector classifier, and the best matched svm-breast-cancer's.
+    "model-selection-breast-cancer": 0.014066138798323302,
 }
 
 SPACES: dict[str, tuple[Callable[..., float], list]] = {
@@ -125,9 +170,30 @@ SPACES: dict[str, tuple[Callable[..., float], list]] = {
 
 
 def ml_problems() -> list[Problem]:
-    """The machine learning tuning problems. Needs scikit-learn."""
+    """The machine learning tuning problems. Needs scikit-learn.
+
+    model-selection-breast-cancer comes twice: with conditions, and as model-selection-breast-cancer-flat
+    with every setting always searched, as before conditions, to show what they are worth.
+    """
     _require_sklearn()
-    return [
+    problems = [
         Problem(name=name, function=function, search_space=SearchSpace(parameters), optimum=OPTIMA[name], tags=frozenset({"ml"}))
         for name, (function, parameters) in SPACES.items()
     ]
+    optimum = OPTIMA["model-selection-breast-cancer"]
+    return problems + [
+        Problem(
+            name="model-selection-breast-cancer",
+            function=model_selection_breast_cancer,
+            search_space=SearchSpace(MODEL_SELECTION, conditions=MODEL_CONDITIONS),
+            optimum=optimum,
+            tags=frozenset({"ml", "conditional"}),
+        ),
+        Problem(
+            name="model-selection-breast-cancer-flat",
+            function=model_selection_breast_cancer,
+            search_space=SearchSpace(MODEL_SELECTION),
+            optimum=optimum,
+            tags=frozenset({"ml", "conditional", "flat"}),
+        ),
+    ]